compressibility check to determine quality-based single-pass?

[theeht]

I've done some searching to try to figure this out but haven't come up with anything I can understand. Is there some way to use the results of the compressibility test to help determine a good setting for the 1-pass quality-based setting in Divx 5? For me knowing file size is desirable but not crucial--but I'd like to have some idea of the likely result, and with my slow system I'd love to eliminate the second pass. If this is covered in some guide, please post a link, otherwise if anyone has the patience to explain to a non-expert I'd much appreciate it.

[jeremymacmull]

hey sorry this is not an answer to your question just another question

What system do you have ?

i have a P3 850 256 mb ram 140 gb hdd Geforce 2 Go 32mb laptop

I got annoyed with how long DVDs were taking to encode with 2 pass so i learnt how to install GKnot with YV12 and got about a 5-8 FPS increase in speed ie on second pass from 12-15 to 20-22 me was very happy after that but it did need a lot of fiddling to get working

if you want to speed it up but still take advantage of 2 pass use YV12!! tho not for the faint hearted!!!

JEREMY

[Sephiros]

Hi Theeth!


The compressibility test is in fact meant to compare the average frame size to a given birate and the average frame if encoded at quality based 100% (quant 2). The value Gknot gives you is the difference percentage between them two.

For example, if the average frame size at 800kbps is 6500 and the quality based gives you an average of 10000, the resulting comp test will be 65%.

So you can work the same way doing a manual comptest as written here by Jonny:
http://forum.doom9.org/showthread.php?s=&threadid=44414

but changing few steps.

1) Proceed as indicated to step 5. take note of the average Delta Size frame.

2) Start again from step 1, but in stand of selecting quantizer 2 (or 100%), select what you want. take note of the average delta size frame at step 5.

you must have a lower value than in the first pass. Divide it with the first value, multiply by 100 and voila! you have your compressibility!


As for the predicted size for a given quantizer, Jonny also indicates how to do that in the thread.


I hope I wasn't too confusing :)


Edit:

I said in the first paragraph that it is the way Gknot proceed...
well not exactly, but it's the idea. Gknot uses instead the predicted video size calculated with the help of the bitrate calculator. But all in all, it's the same thing :)

[theeht]

Originally posted by jeremymacmull
hey sorry this is not an answer to your question just another question

What system do you have ?

i have a P3 850 256 mb ram 140 gb hdd Geforce 2 Go 32mb laptop

[...] if you want to speed it up but still take advantage of 2 pass use YV12!! tho not for the faint hearted!!!

JEREMY
System is P3 550 mhz, 384 mb ram, Win98 SE. Thanks for the suggestion, but I haven't a clue what YV12 is, and I'm probably faint of heart here.

[theeht]

Thank you for the link to that discussion--the equation that predicts the file size for a given quality-based single-pass setting is helpful. I didn't understand why it would be necessary to do a manual compressibility test rather than the one built in to Gordian Knot or, to get back to my original question, how one might link the results of a compressibility test to the likely file size of the encoded video.

[Sephiros]

I told you: it's because it uses the predicted file size. so what you can do as an alternative, is find the predicted file size in KB using the method described by Jonny, and then enter the value in Gknot.
then make a comptest normally (if your ae not with 5.03, because it's not supported by Gknot)

[jeremymacmull]

YV12 is the colourspace that DVDs and DivX are natively stored in Gknot and others often convert to another colourspace before encoding this conversion takes time and processor power to do

so to give us a speed boost it skips the conversion step and keeps all colourspaces as YV12 definately not for u then if you are faint of heart as it took me ages to get working (but i was probably doing a whole lot of things wrong to begin with)

but it did solve my speed probs (now a 2 pass full movie takes 6hrs instead of 9-11hrs)

on a side note why do you have such slow system its quite cheap to upgrade nowadays is it not :P

JEREMY

[theeht]

Originally posted by jeremymacmull
on a side note why do you have such slow system its quite cheap to upgrade nowadays is it not :P

because I have a Dell. Can't swap the motherboard... therefore no longer so cheap (could buy many dvd's for the price of a new system).

[theeht]

Originally posted by Sephiros
I told you: it's because it uses the predicted file size. so what you can do as an alternative, is find the predicted file size in KB using the method described by Jonny, and then enter the value in Gknot.
then make a comptest normally (if your ae not with 5.03, because it's not supported by Gknot)
I'm sorry, I'm sure a smarter person would understand these issues more easily... I find the predicted file size by multiplying the average delta frame size by the total number of frames. I enter this number in GKnot--where? Then make a compressibility test which tells me--what (how do I use the resulting numbers)? And I learn the likely file size at a given one-pass setting--how? I'm sorry, obviously I have little grasp over the core issues (this is why Gknot appeals to me), but if you want to give up on me I'll understand.

[OvERaCiD23]

Originally posted by jeremymacmull
...as it took me ages to get working (but i was probably doing a whole lot of things wrong to begin with)

but it did solve my speed probs (now a 2 pass full movie takes 6hrs instead of 9-11hrs)


All it takes is uninstalling AviSynth 2.0x and then installing 2.5.x. What troubles do you run into?

The speed increase is quite dramatic; I also encode on a 900mhz laptop, and I got about the same speed jump as you stated. Pictures seem to be a bit sharper, but I've only done 5 movies now in YV12, so it could just be in my head.

[Sephiros]

Originally posted by theeht
I'm sorry, I'm sure a smarter person would understand these issues more easily... I find the predicted file size by multiplying the average delta frames by the total number of frames. I enter this number in GKnot--where? Then make a compressibility test which tells me--what (how do I use the resulting numbers)? And I learn the likely file size at a given one-pass setting--how? I'm sorry, obviously I have little grasp over the core issues (this is why Gknot appeals to me), but if you want to give up on me I'll understand.


You got the first part right :)
I just realise that Gknot doesn't allow you to enter a given video-size.

You'll have to do as I said in my first post. (One pass quality based 100% compared to one pass to the quality based you want)

As the for the result, it tells you that the movie will be at XX% at the given quality of the quality it would be if encoded at 100%.
roughly, if the comp. test is 50%, then the quality you'll get is divided by 2 compared to the original.

[theeht]

Originally posted by Sephiros
You got the first part right :)
I just realise that Gknot doesn't allow you to enter a given video-size.

You'll have to do as I said in my first post. (One pass quality based 100% compared to one pass to the quality based you want)

But this is what I don't understand--the second value, "the quality-based level I want," is precisely what I am trying to determine. I'd use 100% if I knew it would fit on 2 cd's, or maybe even three, but not four...


As the for the result, it tells you that the movie will be at XX% at the given quality of the quality it would be if encoded at 100%.
roughly, if the comp. test is 50%, then the quality you'll get is divided by 2 compared to the original.


Thank you again for your efforts, but I'm sorry, I'm just not getting it--how do these steps get me to my goal? It looks like this process determines a movie's "compressibility"--but a) I don't understand why this is better than GKnot's built-in test, which also determines compressibility, and b) I don't understand how this tells me what quantizer setting will produce what file size. I see now that one can save several avi files using different quality levels and multiply average delta frame size by no. of frames in each, but that's a lot of time and effort--I'm trying to find a simpler way, even if it is not very exact.

To reiterate, I'm wondering if there is some way using GKnot to approximately predict what file sizes various one-pass quality-based settings would produce. Something I read somewhere implied that the compressibility test could be used in such a process, but I don't understand how.

[Sephiros]

Ok now I understand what you mean :)

the comp test helps to set the output quality we want. So i guess (but that's just gessing, I never worked with 1 pass quality based except for comp tests) that if you roughly want the equivalent of a comp test of 65%, you got to do a quality based pass @ 65%.

As for extrapolating file size with precision, I've been doing a lot of experiments and I must tell you that is almost impossible without doing a test each time, for each different setting/resolution. It follows some kind of Cubic equation, but once again, the parameters vary from a movie to another, depending of the resolution etc.

Manualy (the only way if you are with divx 5.0.3):

So what I suggest is that you make a manual comp test @ 100% to get the predicted file size @ 100% (avg D frames size * total number of frame).
let say it gives you 3000 Mb (3 Gb)
You want the movie on 2 CD? 1400 / 3000 = 47%.
Make a the quality based pass @ 47%.

In Gknot ( divx < 5.0.3)

-Do a comp test to a given resolution (let say it gives you 67%)
-Divide the video size given by gknot by the comp test value (let say 650Mb / 0.67 = 970Mb)
-970Mb is how fat your movie will be @ 100% to the given resolution.

as you can see my supposition is that Gknot comp test value "should" give you the % of the quality based pass to reach a given size...

But this is far fetched, and It's beyond my knowledge. Some experts here should know way better than me. And please correct me if Im wrong :p
Unfortunatly, It's all I can do for you ... :(

edit:
I forgot to mention that Divx5Enc, from Jonny, has the option to calculate the predicted size to a given quantizer. Maybe you should try it...

[theeht]

Originally posted by Sephiros
Ok now I understand what you mean :)

So what I suggest is that you make a manual comp test @ 100% to get the predicted file size @ 100% (avg D frames size * total number of frame).
let say it gives you 3000 Mb (3 Gb)
You want the movie on 2 CD? 1400 / 3000 = 47%.
Make a the quality based pass @ 47%.

I see... the problem with this is that quality-based at 47% will, I believe, look much worse than the two-pass.

In Gknot ( divx < 5.0.3)

-Do a comp test to a given resolution (let say it gives you 67%)
-Divide the video size given by gknot by the comp test value (let say 650Mb / 0.67 = 970Mb)
-970Mb is how fat your movie will be @ 100% to the given resolution.


Great! that gives me something to work with.

I forgot to mention that Divx5Enc, from Jonny, has the option to calculate the predicted size to a given quantizer. Maybe you should try it...
Perhaps--when I looked at it I was put off by the need to install particular versions of VirtualDubMod/Nandub and other stuff--again, this is why GordianKnot appeals to me, everything bundled together.