"compressibility percentage", "comptest value", "expected quality of first pass size"

[krism]

Hello there. I've recently started using AutoGK, getting what seems excellent XviD results using the parameters AutoGK suggests. Ultimate goal is to convert my collection of DVD's and TV series to XviD and dump everything on a 500 (or by then 1TB) HD, as we move house and country quite a bit.

Anyway... I have been reading tutorials on the Internet and have been looking at the log reports and there's a few things I don't quite understand. So here goes:

I converted an old DVD this afternoon, it's two comedians on old TV recordings (put on DVD and sold by the TV station). I get the following log:


----- quote log report -----
[27/01/2007 13:53:21] AutoGK 2.40
[27/01/2007 13:53:21] OS: Win2000 (5.0.2195).2
[27/01/2007 13:53:21] Job started.
[27/01/2007 13:53:21] Input file: C:\Gaston & Leo VOLUME 2\VIDEO_TS\VTS_01_0.IFO
[27/01/2007 13:53:21] Output file: C:\Gaston & Leo VOLUME 2\VIDEO_TS\Gaston & Leo VOLUME 2.avi
[27/01/2007 13:53:21] Output codec: XviD
[27/01/2007 13:53:21] Audio 1: Nederlands AC3 2ch
[27/01/2007 13:53:21] Subtitles: none
[27/01/2007 13:53:21] Format: AVI
[27/01/2007 13:53:21] Target size: 1120Mb
[27/01/2007 13:53:21] Audio 1 settings: CBR MP3 with bitrate: 192Kbps
[27/01/2007 13:53:21] Started encoding.
[27/01/2007 13:53:21] Demuxing and indexing.
[27/01/2007 13:59:35] Processing file: C:\Gaston & Leo VOLUME 2\VIDEO_TS\VTS_01_1.VOB
[27/01/2007 13:59:35] Processing file: C:\Gaston & Leo VOLUME 2\VIDEO_TS\VTS_01_2.VOB
[27/01/2007 13:59:35] Processing file: C:\Gaston & Leo VOLUME 2\VIDEO_TS\VTS_01_3.VOB
[27/01/2007 13:59:35] Processing file: C:\Gaston & Leo VOLUME 2\VIDEO_TS\VTS_01_4.VOB
[27/01/2007 13:59:35] Source resolution: 720x576
[27/01/2007 13:59:35] Found PAL source.
[27/01/2007 13:59:35] Source aspect ratio: 4:3
[27/01/2007 13:59:36] Analyzing source.
[27/01/2007 14:04:25] Source has percentage of interlacing in motion areas: 83,83
[27/01/2007 14:04:25] Source is considered to be interlaced.
[27/01/2007 14:04:25] Output will contain 135293 frames
[27/01/2007 14:04:25] Decoding audio.
[27/01/2007 14:06:21] Normalizing audio.
[27/01/2007 14:09:57] Encoding audio.
[27/01/2007 14:21:34] Audio1 size: 129,882,816 bytes (123.87 Mb)
[27/01/2007 14:21:34] Overhead: 1,606,656 bytes (1.53 Mb)
[27/01/2007 14:21:34] Video size: 1,042,915,648 bytes (994.60 Mb)
[27/01/2007 14:21:34] Running compressibility test.
[27/01/2007 14:21:34] Writing the following script to C:\Gaston & Leo VOLUME 2\VIDEO_TS\agk_tmp\Gaston & Leo VOLUME 2_comptest.avs
===========================================================
LoadPlugin("Z:\PROGRA~1\AutoGK\DGMPGDec\DGDecode.dll")
LoadPlugin("Z:\PROGRA~1\AutoGK\filters\autocrop.dll")
LoadPlugin("Z:\PROGRA~1\AutoGK\filters\RemoveGrainSSE2.dll")
LoadPlugin("Z:\PROGRA~1\AutoGK\filters\leakkerneldeint.dll")

movie = mpeg2source("C:\Gaston & Leo VOLUME 2\VIDEO_TS\agk_tmp\Gaston & Leo VOLUME 2.d2v")
cropclip = autocrop(movie,mode=0,wmultof=4,hmultof=4,samples=10,aspect=0,threshold=34,samplestartframe=0,leftadd=0,rightadd=0,topadd=0,bottomadd=0)
fixed_aspect = 1.06666666666667
c_width = width(cropclip)
c_height = round(height(cropclip) / fixed_aspect)
input_par = float(c_width)/float(c_height)
input_par = (input_par > 1.4) || (input_par < 1.25) ? input_par : (4.0/3.0)
out_width = 704
out_height = round(float(out_width) / input_par)
hmod = out_height - (floor(out_height / 16 ) * 16)
out_height = (hmod > 4) ? (out_height + (16 - hmod)) : (out_height - hmod)
new_aspect = (float(out_width) / float(out_height)) / fixed_aspect
LeakKernelDeInt(movie,order=1,sharp=true)
autocrop(mode=0,wmultof=4,hmultof=4,samples=10,aspect=new_aspect,threshold=34,samplestartframe=0,leftadd=0,rightadd=0,topadd=0,bottomadd=0)
LanczosResize(out_width,out_height)
RemoveGrain(mode=2)
SelectRangeEvery(300,15)
===========================================================
[27/01/2007 14:32:39] Duration was: 11 minutes 5 seconds
[27/01/2007 14:32:39] Speed was: 10,16 fps.
[27/01/2007 14:32:40] Compressibility percentage is: 28,76
[27/01/2007 14:32:40] Using softer matrix.
[27/01/2007 14:32:40] Chosen resolution is: 512x384 ( AR: 1,33 )
[27/01/2007 14:32:40] Predicted comptest value is: 72,41%
[27/01/2007 14:32:40] Running first pass.
[27/01/2007 14:32:40] Writing the following script to C:\Gaston & Leo VOLUME 2\VIDEO_TS\agk_tmp\Gaston & Leo VOLUME 2_movie.avs
===========================================================
LoadPlugin("Z:\PROGRA~1\AutoGK\DGMPGDec\DGDecode.dll")
LoadPlugin("Z:\PROGRA~1\AutoGK\filters\autocrop.dll")
LoadPlugin("Z:\PROGRA~1\AutoGK\filters\RemoveGrainSSE2.dll")
LoadPlugin("Z:\PROGRA~1\AutoGK\filters\leakkerneldeint.dll")

movie = mpeg2source("C:\Gaston & Leo VOLUME 2\VIDEO_TS\agk_tmp\Gaston & Leo VOLUME 2.d2v")
cropclip = autocrop(movie,mode=0,wmultof=4,hmultof=4,samples=10,aspect=0,threshold=34,samplestartframe=0,leftadd=0,rightadd=0,topadd=0,bottomadd=0)
fixed_aspect = 1.06666666666667
c_width = width(cropclip)
c_height = round(height(cropclip) / fixed_aspect)
input_par = float(c_width)/float(c_height)
input_par = (input_par > 1.4) || (input_par < 1.25) ? input_par : (4.0/3.0)
out_width = 512
out_height = round(float(out_width) / input_par)
hmod = out_height - (floor(out_height / 16 ) * 16)
out_height = (hmod > 4) ? (out_height + (16 - hmod)) : (out_height - hmod)
new_aspect = (float(out_width) / float(out_height)) / fixed_aspect
LeakKernelDeInt(movie,order=1,sharp=true)
autocrop(mode=0,wmultof=4,hmultof=4,samples=10,aspect=new_aspect,threshold=34,samplestartframe=0,leftadd=0,rightadd=0,topadd=0,bottomadd=0)
LanczosResize(out_width,out_height)
RemoveGrain(mode=2)
===========================================================
[27/01/2007 15:55:45] Duration was: 1 hour, 23 minutes 5 seconds
[27/01/2007 15:55:45] Speed was: 27,14 fps.
[27/01/2007 15:55:46] Expected quality of first pass size: 79,07%
[27/01/2007 15:55:46] Running second pass.
[27/01/2007 17:54:46] Duration was: 1 hour, 58 minutes 59 seconds
[27/01/2007 17:54:46] Speed was: 18,95 fps.
[27/01/2007 17:54:46] Job finished. Total time: 4 hours 1 minute, 24 seconds
----- end quote log report -----


Here's the bits that interested me:


----- quote log report -----
[27/01/2007 13:53:21] Target size: 1120Mb
[27/01/2007 13:53:21] Audio 1 settings: CBR MP3 with bitrate: 192Kbps
[27/01/2007 14:21:34] Audio1 size: 129,882,816 bytes (123.87 Mb)
[27/01/2007 14:21:34] Overhead: 1,606,656 bytes (1.53 Mb)
[27/01/2007 14:21:34] Video size: 1,042,915,648 bytes (994.60 Mb)
[27/01/2007 14:21:34] Running compressibility test.

[27/01/2007 14:32:40] Compressibility percentage is: 28,76
[27/01/2007 14:32:40] Using softer matrix.
[27/01/2007 14:32:40] Chosen resolution is: 512x384 ( AR: 1,33 )
[27/01/2007 14:32:40] Predicted comptest value is: 72,41%
[27/01/2007 14:32:40] Running first pass.
[27/01/2007 15:55:46] Expected quality of first pass size: 79,07%
[27/01/2007 15:55:46] Running second pass.
[27/01/2007 17:54:46] Job finished. Total time: 4 hours 1 minute, 24 seconds
----- end quote log report -----

Now, can anyone help me understand how to interpret these numbers?

Maybe a pretty obvious question to you XviD expers, but could you in plain English explain what exactly these numbers mean and especially how I should interpret them (I did a lot of googling on this the past few days but didn't find anything that explains clearly what exactly they are)

-compressibility percentage
-comptest value
-expected quality of first pass size


In this example, the compressibility percentage is 28,76. Does this mean the original movie is very difficult to compress (seems a rather low percentage). However, the 'expected quality of first pass size' is 79,07 which seems rather excellent. I'm puzzled.

I'd be most grateful if anyone could give a bit of insight. Ta!

[The Scientist]

Yes it can mean it's difficult to compress, more specifically, the compression at your current settings. It is using a softer matrix to improve compressibility hence the value rising.

If you're not happy with the original value, or the finished encode, you can do a couple of things.

Lower the resolution: Probably not recommended as you are already at 512.

Higher the file size: Go to 1493 (3 on a DVDr)

You could lower the audio bitrate to 128 to give you a few more bits for the video.

It's just unfortunate you have a full frame 4:3 interlaced source, it will need more bits than your traditional widescreen progressive movie.

[manono]

Hello and welcome to the forum,

-compressibility percentage I'm supposed to be an expert, but I'm a little hazy on some of the particulars at this point, so someone correct me if I say anything stupid. AutoGK runs a compress test (of 14 frames in every 280, or 5% of the entire video) to help it choose the proper resolution and quality for the encode, given any parameters you've set (in this case, file size of 1120 MB (1/4 DVD size?) and 192 audio). It's a 4:3 DVD, so the test is run at 640x480 (I think), and it returns a value of 28.76%. Way too low. That means it'll be 28.76% the size of an encode done for quant 2 (=100%). AutoGK aims for something like 70-75%. So, it juggles some things to give you better numbers.

-comptest value After lowering the resolution and switching to a lower bitrate quantisation matrix, it figures the comptest value should be 72.41%, just what it wants. This is just an estimate, though, based on the results of the first test.

-expected quality of first pass size And after running the first pass on the complete video, it turns out the estimate was a bit low, and the real percentage value will be 79.07%. Nothing wrong with that.

And The Scientist beat me to it.

[krism]

Thank you, Scientist and Manano for your quick help.


Hello and welcome to the forum,


Thank you!



-compressibility percentage I'm supposed to be an
expert, but I'm a little hazy on some of the particulars at this point, so someone correct me if I say anything stupid. AutoGK runs a compress test (of 14 frames in every 280, or 5% of the entire video) to help it choose the proper resolution and quality for the encode, given any parameters you've set (in this case, file size of 1120 MB (1/4 DVD size?) and 192 audio). It's a 4:3 DVD, so the test is run at 640x480 (I think), and it returns a value of 28.76%. Way too low. That means it'll be 28.76% the size of an encode done for quant 2 (=100%).


So does this mean AutoGK uses its 'default initial parameters', combines this with my own settings (1/4 DVD and 192 audio) and calculates the result would be 28.76%, there were 100% is a 'perfect' copy? Correct me if I'm wrong!


AutoGK aims for something like 70-75%. So, it juggles some things to give you better numbers.


So after seeing the rather low 28.76% AutoGk decides it needs to change some parameters to increase that low percentage, so it lowers the resolution, does some other stuff (a 'softer matrix' as The Scientist suggest),... Correct?



-comptest value After lowering the resolution and switching to a lower bitrate quantisation matrix, it figures the comptest value should be 72.41%, just what it wants. This is just an estimate, though, based on the results of the first test.


So after changing the parameters, AutoGK runs the test it performed before again, this time with the adjusted parameters, and comes up with 72.41%. It then starts the 'real' encoding process using these parameters. Correct?



-expected quality of first pass size And after running the first pass on the complete video, it turns out the estimate was a bit low, and the real percentage value will be 79.07%. Nothing wrong with that.


So AutoGK completely encodes the movie with the adjusted parameters and after enconding concludes the real percentage was 79.07%


Does a "expected quality of first pass size" of 100% mean an 'absolutely perfect' copy, absolutely 100% the same as the original DVD where absolutely no difference between audio and video can be detected?


Another question, why is there a second pass??


Thank you for your help!

[krism]

So if I try and analyse what you told me then:

-compressibility percentage would be the predicted result of an encode using the parameters I set (1/4 DVDR, 192 audio) before AutoGK makes any adjustements to its encoding parameters.

-comptest value would be the predicted result after AutoGK makes some adjustments to increase the original too low "compressibility percentage"

-expected quality of first pass size would be the actual, real result AFTER encoding.

Correct me if I'm wrong (which has been known to happen once or twice - an hour ;-)

[manono]

Hi-

So does this mean AutoGK uses its 'default initial parameters', combines this with my own settings (1/4 DVD and 192 audio) and calculates the result would be 28.76%, there were 100% is a 'perfect' copy? Correct me if I'm wrong!

Well, as good as you're going to get with the default settings AutoGK uses for its tests.

...there were 100% is a 'perfect' copy? Correct me if I'm wrong!

Yes, 100% is as good as it gets. Sometimes people allow too big a size, and the results come out at over 100%, and it'll raise the resolution and can do some other things, but it can't always reach a really large size.

So after changing the parameters, AutoGK runs the test it performed before again

No, it just ran the compress test once, and extrapolated from the results to "guess" what the results would be for 512x384 and the softer matrix.

So AutoGK completely encodes the movie with the adjusted parameters and after enconding concludes the real percentage was 79.07%

Yes, the real percentage after the first of 2 passes.

Does a "expected quality of first pass size" of 100% mean an 'absolutely perfect' copy, absolutely 100% the same as the original DVD where absolutely no difference between audio and video can be detected?

No, any reencode lowers the quality. It's good, but not as good as the source.