Compressibility check question

[iparout]

Hi all.

I've been using DivX 5.05 Pro and GKnot 0.28.5 for about a month now, and I have noticed a big difference with DivX 5.02 Pro and GKnot 0.26 which I had been using before. The difference is with the compressibility checkes. In the past, a compressibility check of 50% yielded very good quality and a test over 85% resulted in undersized files. With DivX 5.05 Pro and GKnot 0.28.5, a compressibility of 50-55% yields a non-acceptable (low) quality, whereas compressibility checks of even 160% (!) do not produce undersized files.

So,what I would like to ask is if there is a difference in calculating the compressibility with the previous combo (DivX 5.02 Pro + GKnot 0.26) and the current combo I am using.

Thanks in advance.

[iparout]

I just did another movie at 101% compressibility check (1CD = 700 MB) and it didn't turn out undersized either. Is anyone else experiencing the same thing ? Maybe I am doing something wrong with the settings of GKnot ?

UPDATE:After a lot of research I think I found out what the problem is. I use DivX Pro 5.0.5 Pro as I have already stated, and this version of the codec has Quantizer = 1 as the 100% quality in 1-pass Quality based method. All previous versions had quantizer = 2 for 100% quality, as far as I can recall. So I guess that GKnot does the quality based 1-pass with a quantizer=2 instead of 1 which leads to a very big % in compressibility test. I have used the tool Enc to figure that out. When I set Quantizer = 2 and do a compressibility check, it comes out at 106,1% for a specific movie, which is the same as the % in GKnot compressibility check. However, when I use quantizer = 1 in Enc, the compressibility check for the same movie (with the exact same settings) drops to 39.6%, which seems more appropriate and correct.

Which are the correct settings for a compressibility check guys ?

[jonny]

Since comp. test & quantizers are related, you can use what you prefer.
The only difference between q=1 & q=2 is that you'll probably target to a different % value (example: using q=2 your target could be 80%, using q=1 your target could be 35%).

[cordraconis]

Originally posted by jonny
Since comp. test & quantizers are related, you can use what you prefer.
The only difference between q=1 & q=2 is that you'll probably target to a different % value (example: using q=2 your target could be 80%, using q=1 your target could be 35%).

I had the same problem a few days before when i did a comp test for Gladiator.
I ended up using and testing different filters (from pure Lanczos to neutral bicubic + Convolution 3D, which was my final choice.)
The difference between pure lanczos and the 2nd option, was only 400mb. (from 3.7 gb to 3.3gb).

After that, I realized the Q1-problem. :D

Is there any "easy" relation between the comp test for Q1 and Q2? (It could have saved me a few hours ;) )
And is it possible that the effect of the filters is less clear on a Q1 encode? (400mb seems very little difference to me, especially compared to a 3.7gb file :confused: )

Ok, Gladiator is a very high quality and detailed movie, so C3D shouldn't remove too much noise/details, but still ... I expected more increase in compressibility.

Any thoughts on that?

[iparout]

I installed DivX 5.0.3 Pro instead and did a compressibility check again whih came out 104% but not undersized. So I have begun to think that it's a problem with the new GKnot. Anyone else experiencing the same problem (extraordinary big % in compressibility checks) ?

[jonny]

I'm sure of the undersize problem with DivX up to version 5.0.2
With 5.0.3+ all is possible :) (with some searching you should find experiences from others)
Big comp.test value means only that the movie is easy (i doubt that there is something wrong with GKnot)


>Is there any "easy" relation between the comp test for Q1 and Q2?
quant=1 encode should be 3 time bigger than quant=2 encode (iirc)
anyway you can try by yourself :)

[iparout]

Originally posted by jonny
I'm sure of the undersize problem with DivX up to version 5.0.2
With 5.0.3+ all is possible :) (with some searching you should find experiences from others)
Big comp.test value means only that the movie is easy (i doubt that there is something wrong with GKnot)

If the movie was easy, then a compressibility check of 140% would yield an undersized output file, however, to my amazement, the file was right on target (not undersized), so I figure that GKnot (or DivX 5.0.5 Pro) is calculating the compressibility in a different way than the previous versions.

[jonny]

Probably some quant=1 are used by divx @ comptest=140% (ffdshow can show this)
Nothing had changed in the gknot comptest field.
If you are near to 100%, you could try a quant=2 encode instead a 2pass encode to see if you like it more (i don't know how the divx>502 rc algo work with c.t.>100%, if the quantizer jumps too much, a costant quant encode could look better)

Note: i don't seriously use divx after the 5.0.3 release, those are only ideas (apart from the FACT that GKnot ct is not changed), someone should test all ^^

[Acaila]

Quote from divx.com under "What's changed" for DivX 5.04:
The new RC doesn't use Q=1 frames anymoreIn other words, DivX 5.03 is able to use Quant 1 and Jonny is correct that this explains why you got the 140% compressibility test (because GKnot bases its calculations on quant 2) and why it didn't result in an over- or undersized file (the codec can compensate for undersized files by using huge quant 1 frames).

[iparout]

Originally posted by Acaila
Quote from divx.com under "What's changed" for DivX 5.04:
In other words, DivX 5.03 is able to use Quant 1 and Jonny is correct that this explains why you got the 140% compressibility test (because GKnot bases its calculations on quant 2) and why it didn't result in an over- or undersized file (the codec can compensate for undersized files by using huge quant 1 frames).

I understand what you are saying but, as my first post says, I have also experienced 160% comp. checks with DivX Pro 5.0.5 as well, which also yielded no undersized files. I really dont know what's going on. And it hasn't happened to one movie only but to the majority of movies I encode.

I used to have a rule in the past that over 50% comp. cheks were acceptable, now I can't use this rule anymore.

[Acaila]

Have you checked in ffdshow if the codec still uses quant 1 frames? It's quite easy to do and will tell you if your problem is a calculation error or 'normal'.

[iparout]

How do I use ffdshow ? I have it installed but I don't know how to use it to check for Q1 frames.

[dTb]

Go to the ffdshow configuration, go to the osd section and check the frame mean quantiser option, enable the osd and then watch the video.

I have to say I haven't experienced anything like this, as near as I can tell the comp tests I do today with 5.0.5 are similar to what I used to do with 5.0.2. In other words I haven't experienced such large changes in comp values that you have.

Doing comp tests with enc using 5.0.5 I use q=2, from what I gather there isn't much value using q=1 in general encodes so I don't use it for comp tests.

[iparout]

It works but the quantizer number keeps changing very fast so I can't tell if it goes to 1 or not. Any way to fix that ? Or at least see a log of all the quantizers ?

[iparout]

Ok, I encoded Friday 13 both with 5.0.5 Pro and DivX 5.0.2 Pro using the EXACT same settings. The compressibility check was the same for both codecs (44% for 1CD and 93,3% for 2CD). The difference was that with DivX 5.0.2 Pro, the output file was 70 MB undersize however with DivX 5.0.5 Pro the filesize was right on target (no undersize). I run the DivX 5.0.5 Pro rip with ffdshow and the quantiser never seemed to drop to 1 (2 was the lowest I saw) but as I already said in my previoys post, the number kept changing too fast so I couldn;t see for sure. However, judging by the fact that the 5.0.5 Pro rip wasn't undersized, I believe that Q1 frames were used in that encode. So what I would also like to ask is if Q1 frames are worth it, or whether I should keep the extra 70 MB for 160 kbs mp3 audio (I am currently using 128 kbps).

And BTW, how can I find a Q1 frame in the video, so as to compare it with the same Q2 one ?

[jonny]

@93% the averare quant should be near to 2, so it's hard to notify the presence of quant=1

you can try to encode @ comp.test = 250%-300%, in this case, if you are not undersized, you should notice a lot of Q1s

PS: to compare Q2 vs Q1 you can simply encode a clip in quality based mode

[dTb]

IIRC there's a tool called drfanalyser that will analyse your file and report on the number of frames with each quant.