Divx 5.1 Bitrate/Comptest/Undersized/Confusion

[Mac Sidewinder]

I searched the boards and found 87 posts concerning this subject (undersized files). But everybody seems to have a different view on this. Some say to set the min and max quant to 2 as this overrides the bitrate, some say that the bitrate overrides the quants. Some say that the comptest % is the factor to go by, some say that the bit/pixel is most important.

I have a dvd that no matter what settings I put in, the comptest results are huge, filesize is undersized significantly, bit/pixel values are low. Here is the problem and some of the fixes users have said:

Using latest version of GordianKnot
VirtualDub 1.4.10
Divx 5.1
(b-frames on)
(psychio v's on light)
GMC enabled
bitrate 750
2 pass encode
128 bit mp3 OR 192 ac3 audio
1 cd encode
min quant 2 - max 8
normal bicubic

I ran jonny's comptest on a 101 min dvd (The Relic (a very dark movie)) and Gnot gives me a bitrate of around 750 w 128 bit audio, (at work and can't remember exactly) and at a 512x res I get a whopping 187% with .215 bit/pixel. I have tried the following:

Increased the res to the max @ 704x (any higher and Gnot res indicator turns red. Comptest @ 135% (re-ran it) Bit/pixel drops to .155 Final file size is around 475mb with 128 bit mp3 audio.

I can swap the mp3 audio for the dvd's 192 bit ac3 but that only pushes the bitrate down to 685 and increases the filesize to around 500mb.

My questions are:

I understand that this a extremely compressable video but should I try to get the comptest % down by using a high res thus sacrificing the bit/pixel value OR should I ignore the comptest, stay with a lower res such as 512x with a higher bit/pixel? Doesn't the bit/pixel directly reflect the quality (such as higher bit/pixel mean less macroblocks and better picture)?

Which setting actually has greater control - the quant setting OR the bitrate used, and how does one effect the other using 2 pass encode?

When you run into a video like this (I don't mind the undersize of the file, just want the highest quality possible) steps do you take?

Luckily I haven't found too many dvds that fall into this category. Any thoughts on this confusing matter (to me at least).

Mac :confused:

[khp]

There was some confusion about undersized files when divx5 first came out.

But to the best of my knowledge everyone has agreed, that you will only get significantly undersized files, if the movie is saturated, with frames using the minimum quantizer. The best way to check this is to do a quality based encoding at 100% quality. If you could test this and post your results, that would be great.

Your results does seem a bit disturbing, 475MB including audio with res 704*X, seems extremly low.

I'am 99.9% sure that quantizer settings override bitrate settings. In bitrate controlled mode, the codec will work within the max and min quantizers, and try to achive the desired file size.

Also is you don't mind doing some testing, you could perhaps try encoding with and without the individual Mpeg-4 options, in quality based encoding. I'am under the impression that some of the Mpeg4-options work extremely well for some movies.

[Mac Sidewinder]

I will do a couple of encodes tonight with the following settings:

1. Leave the audio at 128 mp3, same res for each
2. One encode with all mpeg 4 options off and quants at 2 (single pass)
3. One encode with min quant at 2 and max at 8 (2 pass)
4. One encode with each mpeg 4 option (2 pass)

Anything else I should try? This will take awhile, will post results as I go.

Mac

[khp]

Originally posted by Mac Sidewinder
Anything else I should try? This will take awhile, will post results as I go.


I think it would be better to use quality based encoding for all the tests. That way we can use filesize as a reliable preformance measure.

[Acaila]

First, let me commend you for doing such an extensive search around the forums. I really appreciate that. :)

Second, I really have to put this undersized/satured file thing in the FAQ sometime....as soon as I have the time I'll work up some thorough explanation.


Now, for your questions.

1. Leave the audio at 128 mp3, same res for each
2. One encode with all mpeg 4 options off and quants at 2 (single pass)
3. One encode with min quant at 2 and max at 8 (2 pass)
4. One encode with each mpeg 4 option (2 pass)

2. will give you the file you already had, namely 475 MB at 704 resolution.
3. will give you the exact same file as 2. because the codec will still use only quants of 2, no matter how high you set the maximum.
4. will give you an even smaller file than 2. because B-frames will now be encoded at quantzers of 2x2=4. MPEG4 settings aren't invented to make files bigger but smaller, which is exactly not what you want.

I understand that this a extremely compressable video but should I try to get the comptest % down by using a high res thus sacrificing the bit/pixel value OR should I ignore the comptest, stay with a lower res such as 512x with a higher bit/pixel? Doesn't the bit/pixel directly reflect the quality (such as higher bit/pixel mean less macroblocks and better picture)?
The bits/pixel only give you a remote indication of quality. In fact it doesn't say much, because it is dependant on too many other factors, like compressability. My suggestion: forget bits/pixel completely.
In your case you've already seen that using a resolution of 704 still gives you a too small file. What do you think will happen if you reduce the resolution to 512? Bits/pixel doesn't tell you anything here.

Which setting actually has greater control - the quant setting OR the bitrate used, and how does one effect the other using 2 pass encode?
The quant setting defines the range of quants the codec can use to compress the movie. The bitrate setting is what the codec uses to calculate which quant to use on each frame depending on motion and complexity and stuff. But this calculated quant is limited by the range you set. The wider the quant range the closer the codec can reach the target bitrate with the lowest possible quants. If you limit the range too much you'll force the codec to use sub-optimal quants.



My suggestions for your encode:
1. Use 1-pass quality based 100% to encode, not 2-pass.
2. Disable all MPEG4 options.
3. Encode at a resolution of 704x...
4. Use the DVD's original AC3 audio.
5. Use GKnot's Sharp Bicubic resizing. This is (0,0.75) in Avisynth or A=-1 in VDub.
6. Add some extra audio tracks if you want 'em. Like director's comments and stuff.
7. If you still end up with a smaller file than you want, deal with it, coz it ain't gonna get bigger. :)
And it's already at the highest possible quality, so who cares?

[jonny]

Sometime happens, i have put "the others" in a 1 cd @ 704x... with a very impressive quality (and with divx4.12!!!), dark movies are really compressible.

[Mac Sidewinder]

Thanks for taking the time to respond. I would like to trouble you just a little bit more though:

The quant setting defines the range of quants the codec can use to compress the movie. The bitrate setting is what the codec uses to calculate which quant to use on each frame depending on motion and complexity and stuff. But this calculated quant is limited by the range you set. The wider the quant range the closer the codec can reach the target bitrate with the lowest possible quants. If you limit the range too much you'll force the codec to use sub-optimal quants.

1. So I take it to mean that the bitrate will establish the final max file size and the quant changes (by examing the source on the first pass) though whatever range you establish to reach this desired goal, changing from 2 (which is the least amount of compression) to whatever upper limit you set, always using the lowest quant possible as long as the filesize is maintained? Unless you come across a source like mine where everything is so compressable that it uses quant 2 no matter what you set the range to. Which leads me to ask:

1a. If quant 2 is always used, does the bitrate matter. In other words, for my encode, increasing the bitrate is not going to give me a better quality encode OR increase the filesize.

1b. If quant 2 is always used, then there is no difference between 1 pass quality set to 100% OR using 2 pass with a range of quants such as a setting of min=2 - max=8? If this is true then if you get over 100% in the compressability setting you can just do a 1 pass 100% quality encode.

1c. If the quant changes to maintain the final filesize, why do some users maintain that you should limit the quant to around min 2 - max 8? Why not simply set it to something like min 2 - max 30, and let it use whatever it wants? I realize that the higher it goes the less quality you get but if the bitrate is going to limit the final filesize, why not let the quant have as much range as possible?

I realize this is lenghtly but please bear with me a little bit longer

If I want to test the different effects of the MPEG4 options, is it better to do what jonny says and do every encode at 100% quality or use the 2 pass mode?

I've already done 3 tests and will post the results on this type of encode. I think I will also take a regular source (not as compressable) and run the same tests and post them also if that is alright.

Thanks for your time.

Mac

[Acaila]

1a. Correct, if with a certain bitrate the codec already uses only quants of 2, then indeed it's not to get any better/bigger if you use a higher bitrate.

1b. Also correct, 1-pass 100% will give the same output as 2-pass mode in your case.

1c. Three in a row, this is correct too! :D. It doesn't really make much sense, at least from the codec's perspective, to limit the range at all. Setting the max quant to 31 instead of 12 won't give you a worse movie, because the codec probably won't use anything higher than 10 anyway. However most people prefer at least a minimum quality of frames with quants of 8. Personally I hate anything with quants of 6 or higher even (read this (http://forum.doom9.org/showthread.php?s=&threadid=24440) for my way). So the max quant is usually a personal baseline for everyone.

And finally for you last question, yes 1-pass quality mode is the best mode to test what various settings have on filesize/quality, because with 2-pass mode the user inputs a certain filesize (=bitrate)and tries to reach it no matter what. And it has the added bonus of being only 1 pass instead of two :)

[Mac Sidewinder]

Thanks for the quick response. If it is alright with you I am going to run some extensive tests on both my very compressable dvd and a normal one and post the results here. I am most concerned with how the new MPEG4 options affect filesize and quality. I also want to see if there is a difference in how much these affect each, such as normal sources are compressed more, etc.

It will take awhile but hey, I like to know why I am doing something, instead of simply reading a guide and doing what it says.

I read your method on quants and have to confess I am going to have to do some research on the xvid codec procedures to understand exactly what you were discussing. I understand what you were saying (mostly) but don't have any experience with xvid.

And lastly, thanks for the pointers for this particular encode. I am not too concerned with final filesize (as far as HAVING to fill up a cd) but I do want to get the best quality that I can. If that means only having a 500mb file, I'm ok with that.

Mac

[Acaila]

Good luck :D

[Mac Sidewinder]

The results are in on my very compressable dvd as follows:

Source: The Relic Dvd 109mins 16:9 All Comp Tests done @ 5%, all settings same as reference except where noted:

100% quality reference made using the following settings:
- Divx 5.01
- 1 Pass Quality @ 100% (quants of 2)
- Resolution 704x304
- Sharp Bicubic Resizing
- Audio 128Kbs VBR
- No MPEG4 Tools Used
- No Psychovisual Enhancements
- Comp Test Result 89.9%
- Bits/Pixel 0.147
- Final Filesize 606mb

1. BFrames turned on: Comp Test 106% - Filesize 473mb
2. GMC turned on: Comp Test 91.3% - Filesize 596mb
3. BFrames & GMC on: Comp Test - 106.2% - Filesize 471mb
4. Quarter Pixels on: Comp Test - 88.8% - Filesize 607mb
5. BFrames & GMC & Quarter Pixels on: Comp Test - 106.3% FS - 469mb
6. Any Psychovisual Enhs on: Comp Test 89.9% - Filesize - 606mb

Visually I couldn't distinguish (while watching, didn't examine frame by frame) between any of these encodes. All looked excellent EXCEPT for #4 (which was jumpy). The difference between the largest (quality reference, 606mb) and the smallest (#3 & #5) is 135mb. This was the result of BFrames being on. Quite a difference but still looked the same.

Conclusion: Really can't tell much since the source was so compressable. I just wanted to see if visually there was much of a difference between highly compressable sources encoded in quality setting and using the MPEG4 options. Please note that the lowest comp test was 88% and I still got lower than expected (what the bitrate said I should get) filesize. So if you get 80% or higher please follow the guidelines set out by Acaila in this thread to get the best quality possible.

Now comes the real test. I am now in the process of doing the same thing with a "normal" dvd source. Will keep you posted.

Mac

[Acaila]

As expected using any combination of B-frames + whatever increases compression. All the other settings give about the same filesize. B-frames allow you to compress a frame much more with less artifacts. However you don't need them to be compressed more, because you've got enough space.
The other MPEG settings and Psycovisual stuff are meant to make the video look better at a low bitrate, but since you can get away with maximum bitrate (= maximum quality) already there's not much point in using them is there?

Now all you need to do is throw in the AC3 audio instead of an MP3 and you've got a 700MB movie (probably a little bit over, but you can play with that some more) with perfect video and perfect audio.

[Mac Sidewinder]

Now all you need to do is throw in the AC3 audio instead of an MP3 and you've got a 700MB movie (probably a little bit over, but you can play with that some more) with perfect video and perfect audio.

Thats exactly what I did for the final product. Came out to be 702mb. I am now in the middle of conducting a test with a "normal" dvd. I wonder if I will receive the same reduction in filesize using BFrames or if since it won't be so compressable if the reduction will be less? Hmmm we'll see.

Mac