DC precision recommendations

[gizzin]

My understanding of dc precision is that if you don't use it properly you will be wasting bits. So what I go by is 3500 and less 8, 3500 to 6000 9, and anything over 10. So maybe some authors of mpeg2 encoders (hank :) ), Or anyone could offer me some recommendations, and explanations why.

Thanks for your time...

[MrC]

If it can help, this is how AVStoDVD compute DC precision:

Select Case .VideoBitRate
Case Is > 6000
DC_Precision = "10"
Case Is > 4000
DC_Precision = "9"
Case Else
DC_Precision = "8"
End Select

;)

Bye

[ron spencer]

I've wondered the whys as well

[Guest]

It just gives you a tradeoff of reduction of possible posterization effects in smooth areas versus bitrate. There's no general formula. You have to choose what tradeoff to make given your goals and esthetics.

[mikenadia]

Will there be any advantage if it were done at the GOP level?

[Manao]

It should be chosen according to the quantizer, not the bitrate. That way, it would be easily made adaptive per picture, since it would naturally depends on its quantizer which varies (except for constant quantizer, of course...)

[hank315]

It should be chosen according to the quantizer, not the bitrate.
Could you please explain why?

[mikenadia]

if the quantizer determines dc_prec, it could also determine a GOP "appropriate" matrix?
If the quantizer is far from the "desired average quant", change matrix for that GOP.

[ron spencer]

are we splitting hairs here...I mean really, for 8 years I have chosen 10 no matter what...results seem fine.

[gizzin]

are we splitting hairs here...I mean really, for 8 years I have chosen 10 no matter what...results seem fine.

That's good that you've chosen that, and the results seem fine, but thats not the point of this thread.

It just gives you a tradeoff of reduction of possible posterization effects in smooth areas versus bitrate. There's no general formula. You have to choose what tradeoff to make given your goals and esthetics.

Posterization occurs when an image's apparent bit depth has been decreased so much that it has a visual impact. The term posterization is used because it can influence your photo similar to how the colors may look in a mass-produced poster, where the print process uses a limited number of color inks. This effect ranges from subtle to quite pronounced, although one's tolerance for posterization may vary.

So where's the tradeoff then? That's now what I'm trying to understand.

[mikenadia]

Sorry if OT.
I did a small test to measure the cost of going from 8 to 10 (7500 frames and always the same matrix MPEG(?))

Q=2 dc=10 average bitrate: 5793
Q=2 dc=8 average bitrate: 5719 diff: 1.29%

Q=4 dc=10 average bitrate: 5171
Q=4 dc=9 average bitrate: 5147
Q=4 dc=8 average bitrate: 5129 diff: 0.89%

Q=8 dc=10 average bitrate: 2484
Q=8 dc=9 average bitrate: 2455
Q=8 dc=8 average bitrate: 2425 diff: 2.43%

The benefit of higher precision may be (?) equivalent to a better quality matrix.
If this is true and if we think that the best matrix is the one aiming at an average quant of 6 ( we could also use a comparaison of INTRAVLC0,1 and 2 as a decision criteria) , we could say that for Q>6 need for worse precision(dc=8) and/or worse matrix (reduce risk of artifacts) and if Q<6, need for better precision and/or better matrix (more details). May be at the GOP level.

Edit: way OT and way above my head, but may be those decisions (matrix and dc) should modify the target-bit allocation model ( for demanding scenes, trade-off between increasing bitrate or choosing worse matrix....)

[Guest]

So where's the tradeoff then? That's now what I'm trying to understand. Higher DC precision reduces posterizing due to the higher accuracy, but it makes the stream larger.

[Manao]

hank315 : DC precision works like a fixed quantization step for the DC. The quantization step, for a 8bit DC precision, is 128/16. For 11bit precision, it's 16/16. The quantization step for other coefficients, assuming a linear quantization scale, is matrix*quantizer*2/16. Thus, if you want to keep DC and AC quantization steps a bit coherent (as it is for ASP/AVC), you have to reduce the precision when you increase the quantizer (doubling the quantizer would reduce the DC precision by 1, once again for the linear quantization scale).

[hank315]

Sounds reasonable but why should there be a relation between the quantization of the DC or AC coefficients with DC being only the mean value of the block in the frequency domain, it's a rounding issue for DCprec 8-10 with DCprec 11 covering the whole 2048 range.

We feed our encoders with 8-bit YUV data, is it really necessary to raise the nr. of DC levels after the DC transform to 512, 1024 or even 2048, don't know, metrics show no significant difference for low quantizers (high bitrate).
Low bitrate encodes show better metric values for low DCprec values due to the lower bit demand.

[mikenadia]

It seems that the advantage of dc=8 over dc=10 ( only in terms of bitrate for a given Q) is matrix ( and also Q) dependent.
I did a small test (same sample as post #11) with different matrixes.
for the same bitrate (around 2425) the difference was much smaller with a low bitrate matrix (AVAMAT7) than with MPEG.
Q=5; matrix AVAMAT7;
dc=8 average bitrate:2430
dc=9 average bitrate:2439
dc=10 average bitrate:2461 diff : 1.27% instead of 2.43%

Q=5; matrix fox1
dc=8 average bitrate:7083
dc=9 average bitrate:7082.

Appropriate matrix; dc8=dc9 ; otherwise dc8 is the one that needs less bitrate to achieve a given Q.

Edit: for a given bitrate,two ways to decrease Q; increase bitrate or taking a worse matrix. At the extremes (very high or low Q corresponding to outliers in the first pass at CQ), it will take too much of a change in bitrate to affect Q and a change in matrix for that GOP may be preferred ( or modifying the coefficients of the matrix choosen for 1st pass by a scaling factor, depending on the difference (+ and -) "outliing" of the complexity of the GOP compared to the complexity of the movie). I do not think that this is what LUMGAIN does.Depending on the complexity ( at least for the outliers), the GOP will be encoded with a new bitrate and a scaled matrix (both helping each other).

[Manao]

Bitrate alone means nothing. You must compare both bitrate and PSNR.