A question for len0x about quality tuning in AutoGK...

[therealjoeblow]

@len0x,

I've done several tests, and I am *not* saying there's anything wrong, this is just for my own interest and discussion.

The latest test I did last night, I ran AutoGK 1.19 with autowidth settings and did a 2-cd encode with AC3-6CH on a 2h16m movie. AutoGK chose a resolution of 608x352. The final encode looks fine to me. I ran it through 'DivX DRF Analyzer v0.9.5' from

http://www.geocities.com/analyzerDRF/

and it determined that the average DRF quantizer was 3.39, and recommends that I should select a target size of 544x320. So, I reran AutoGK, and left the autowidth turned on, but this time I set the audio to AC3-2CH, assuming that AutoGK would use the same resolution, but use the extra bits to get a lower DRF quantizer. But instead, it now chose a resolution of 640x368, and the resulting DRF quant was 3.41 with a recommended resolution of 576x320, not much change in the DRF Analyzer report.

Both of the final encodes look fine to me, but that's what I've noticed in just about all cases over the past months when I enable autowidth - AutoGK always produces files that DRF Analyzer pegs at average Q~3.4, and it always recommends using a smaller size.

I don't know if my theory has any merit or not, but since encoding video is a lossy process, and you *have* to throw away data, the way I look at it is you have 2 choices:

1) go with a smaller WxH size so that the codec has less pixels to encode, and therefore can allocate more bits per pixel (and so the codec has to do less of the data dumping) - in this case you have to let the resizer throw away the data without a whole lot of control,

*or*

2) go with a larger WxH size, and let the codec do it's magic in throwing away the data.

The way I look at it, the codec *should* be the more intelligent of the 2 (codec vs. resizer), and therefore it's probably better to let it do it's work, since that's what it was designed to. Going with the resizer as the brains behind the data discard operation probably isn't as good an idea, since it can't make intelligent decisions and it just averages out the process. In the resizer case, you probably get a worse *overall* end product, even though DRF Analyzer will tell you that your average quantizer is lower (and therefore the codec had to throw away less data) - The way I'd see it is that the codec had to throw out less data simply because less data existed in the first place to work with, so the DRF analyzer report is somewhat 'artificial'.


What are your thoughts - is that the logic you implemented in your decision scripts?


I've appended the DRF Analyzer Reports for both trials following:


*********************
***FILE 1 ANALYSIS***
*********************

DivX DRF Analyzer v0.9.5 Report!
File Name: J:\Encodes\File1.avi
FourCC: XVID
Codec: XviD0030
Resolution: [ Width: 608 Height: 352 ]
Frame Rate: 23.976 frames per second
The Video has 196032 frames [ 02:16:16 ]

Average Frame quality is MEDIUM [Average DRF/quantizer is 3.39]
Standard Deviation: Quality is MEDIUM [Std. Deviation is 1.00]
Image Resolution is MEDIUM

There are NO frame drops ( NO drops is better )

Recomended Resolution: [544x320] (Target DRF/quantizer=2.8)

Performance Caracteristics:
Macroblocks per frame: 836
The Width is multiple of 32

Kilobits per Second: 958.22
Kilobits per Frame: 39.96
Kilobits per Macroblock: 0.048
Bits per Pixel: 0.19

Frame Type Statistics :
I Frames: 0.80%
P Frames: 50.17%
B Frames: 49.04%
S Frames: 0.00%
N Frames: 0.00%
(More Advanced Codecs use B and S frames)
Frame Quality Statistics :

DRF=1&2: 46005 23.7%
DRF=3: 52335 26.9%
DRF=4: 67696 34.8%
DRF=5: 28431 14.6%
DRF=6: 0 0.0%
DRF=7: 0 0.0%
DRF=8: 0 0.0%
DRF=9: 0 0.0%
DRF>9: 0 0.0%
KeyF/DeltaF: 0.80%
KeyDRF<4: 1565
KeyDRF=4: 0
KeyDRF>4: 0

AverageKeyDRF: 2.44
MAXDRF: 5
AverageDRF: 3.39
Deviation: 1.00



*********************
***FILE 1 ANALYSIS***
*********************



DivX DRF Analyzer v0.9.5 Report!
File Name: J:\Encodes\File2.avi
FourCC: XVID
Codec: XviD0030
Resolution: [ Width: 640 Height: 368 ]
Frame Rate: 23.976 frames per second
The Video has 196032 frames [ 02:16:16 ]

Average Frame quality is MEDIUM [Average DRF/quantizer is 3.41]
Standard Deviation: Quality is MEDIUM [Std. Deviation is 1.00]
Image Resolution is HIGH

There are NO frame drops ( NO drops is better )

Recomended Resolution: [576x320] (Target DRF/quantizer=2.8)

Performance Caracteristics:
Macroblocks per frame: 920
The Width is multiple of 32

Kilobits per Second: 1020.72
Kilobits per Frame: 42.57
Kilobits per Macroblock: 0.046
Bits per Pixel: 0.19

Frame Type Statistics :
I Frames: 0.80%
P Frames: 50.20%
B Frames: 49.00%
S Frames: 0.00%
N Frames: 0.00%
(More Advanced Codecs use B and S frames)
Frame Quality Statistics :

DRF=1&2: 43696 22.5%
DRF=3: 54714 28.1%
DRF=4: 66130 34.0%
DRF=5: 29927 15.4%
DRF=6: 0 0.0%
DRF=7: 0 0.0%
DRF=8: 0 0.0%
DRF=9: 0 0.0%
DRF>9: 0 0.0%
KeyF/DeltaF: 0.80%
KeyDRF<4: 1565
KeyDRF=4: 0
KeyDRF>4: 0

AverageKeyDRF: 2.48
MAXDRF: 5
AverageDRF: 3.41
Deviation: 1.00

[manono]

Hi Joe-

I'm not len0x, but you've stumbled across my only real objection to DRF Analyzer. On the one hand it tells you that More Advanced Codecs use B and S frames, thereby encouraging you to use them. And on the other hand, if you do use them, you will always be told that your chosen resolution is too high, your quality is too low (ave quant), and you should have used a lower resolution. The information that DRF Analyzer gives you is not really wrong, but terribly misleading, unless you understand what's going on. I'll ignore the S Frames, as AutoGK doesn't use them, because there's no standalone today that can play an .avi that uses XviD GMC (S Frames). The problem is in the way that DRF Analyzer interprets B Frames, which AutoGK does use.

AutoGK aims for a 70% or so quality and achieves that by adjusting the resolution, resizer and/or matrix to achieve that. Unless the resolution is in danger of becoming too low, in which case it allows the percentage to slip downwards. That 70% equates roughly to an average quant of 2.85, and is close to the DRF Analyzer guideline of 2.8 before B frames are taken into account.

In the earlier versions of XviD, AutoGK used 1 B frame, a quantizer ratio of 150, and an offset of 100, or 1/150/100. With the new 1.0 RC4, this becomes 1/1.5/1. Since B frames are based on the P frames on either side, then with P frames of 2 and 2, the B Frame becomes 4. With P frames on either side of 2 and 3, the B frame becomes 4 (averaged down from 4.75), and with P Frames of 3 and 3, the B Frame in the middle is 5 (averaged down from 5.5). There's more information on this and other things in Crusty's excellent XviD FAQ:

http://www.vslcatena.nl/~ronald/docs/xvidfaq.html#C3j

A B Frame of, for example, 5, isn't nearly as bad as a P frame of 5. But DRF Analyzer doesn't take that into account. Yes, it's not quite the same quality as the P Frames on which it's based, but the compression gain with B Frames is so great that it's silly not to use them, in my opinion. If you want to get an idea of the true quality of the encode, then try and break out the P and I Frame quants. In your first example, the P Frames are 50.17% of the total, and the I Frames (keyframes) are .80%, for a rough total of 51%. You have 23.7% quant 1 and 2 P Frames, and 26.9% of quant 3 P Frames. All the rest are B frames. The P and I Frames total 50.6%, close enough. That gives an average quant of roughly 2.53 and a quality percentage of roughly 79%. My math may be a bit off, but you get the idea. And that gives you a much better idea of the quality of the encode.

In addition, the XviD 1.0 versions are more compressible than the old versions. I don't know what the expected quality of the first pass size was in your log, but I would guess it to be 5-8% less than the 79% I just gave you. I expect that len0x will take that into account at some point.

...but this time I set the audio to AC3-2CH, assuming that AutoGK would use the same resolution, but use the extra bits to get a lower DRF quantizer.

Nah, it doesn't do that. It aims for a similar quality for all the encodes, based on the results of the compress test, unless the resolution is in danger of going too low. You freed up more space by using the 2 channel AC3, so it upped the resolution.

AutoGK always produces files that DRF Analyzer pegs at average Q~3.4,

Yep, that's what it does alright. You've proven that AutoGK is doing a good job at what it sets out to do. That 3.4 with B frames would be analogous to quant 2.4-2.7 or so, not counting the B-Frames. I gave a fairly wide spread, because the figure will vary depending on a number of factors, but especially the percentage of B Frames. I've seen the B frame percentage go as low as 38% with those settings, and my average is around 46-48%. I would guess that the movie with which you tested is a low-action movie, and the percentage of B Frames is especially high. Again, I believe that the final figures with the RC4 are now better than before, because of the added compressibility of these new versions of XviD.

And if I've said anything wrong, then I hope len0x or someone else will step in and set me straight. Sorry for the long post.

[yotsuya-san]

You can also disable the b-frame "support" in DrfAnalyzer changing the option in its ini file "PACKETED_AVI_SUPORT" from 1 to 0. After this, it does not take into consideration the low quantizer of b-frame in the Average DRF/quantizer.

[manono]

You can do that? Cool. Thanks for the tip, yatsuya-san.

Wait a minute-I just tried it and whether I set PACKETED_AVI_SUPORT to 0 or to 1, I get the same result for the ave quant. It looks to me like that only affects whether or not Packed Bitstream is detected. Or were you talking about something slightly different?

I'm using VERSION = 0.9.5.0.

[yotsuya-san]

Just tried on an avi...

with PACKETED_AVI_SUPORT = 0: DRF Average = 3,741;
with PACKETED_AVI_SUPORT = 1: DRF Average = 5,451

:confused:

Maybe we are talking about different things?

[len0x]

may be I should finally try DRF analyzer... :)

[manono]

Hi-

I just tried again, and I get the same result both ways. Are you actually using Packed Bitstream? I don't, and maybe using it makes the difference in our results.

[yotsuya-san]

Sorry for my ignorance, manono, but I don't even know what "packed bitstream" is...:scared:
When I find something about it through guides and this forum, I can tell if I have used it before ;)

[manono]

Hi-

If you're encoding through AutoGK, then Packed Bitstream isn't used. But if you encode through GKnot, or directly in VDubMod, then it's a default setting (which should be turned off, I think). If you're using it, then make sure you have PACKETED_AVI_SUPORT = 1, and you'll see something about Packeted Frames being used, in DRF Analyzer. If you don't use it, then it doesn't say anything at all about it.

Here's some information about Packed Bitstream:

http://forum.doom9.org/showthread.php?s=&threadid=72903&highlight=Packed+bitstream