Compressibility definition ?

[Poutnik]

How is defined compressibility of a movie ?

I read about it many times, having rough idea what it is,
but in spite of many searches have never found its definition.

BTW I cannot force forum to search by AND logic.(now, worked before I think)
Even "+A +B" and "A AND B" gives results "ORed" according to result page.

[celtic_druid]

Well I guess one definition would be the size per a given quant. So a compressible movie will have a smaller size per quant, giving you a lower average quant for a set size. A movie that is hard to compress will be larget per quant and as such require a large size/bitrate to hit the same average quant as the compressable movie.

[DarkZell666]

giving you a higher average quant for a set size.


shouldn't it be a lower average quant for a set size ? / higher quality at same filesize ?

Let's try to put it simpler :

ex:
A movie which is hard to compress @ q2 will give you >2GB
A movie which is easy to compress (compressible) @ q2 will give you <= 700MB (this happened to me once with "final destination 2", I just did a quant2 encode w/ bframes and it fitted perfect on 700MB with the audio first shot, lol)

So, if you are trying some avs scripts or the like to make the movie more compressible (ie: cleaner in most cases), just take a couple of samples, encode them @ q2, and see which gives the smaller filesize.

[celtic_druid]

Exactly what I meant. Too many higher/lower, smaller/larger's in one paragraph.

[Poutnik]

I understand quite well, what compressibility is about.
But it should be independent on file size, so does liquid density
measuring is independent on volume of measuring bottle.
What e.g. means compressibility is 60 % ?

It should be dependent on movie properties only ( by strict given test procedure) and independent on testing subject ( = me and what I am going to do with a clip ).

[Kopernikus]

I don`t think there is a strict definition of compressibility. There are too much parameters involved. It depends not only on the source, but on the format (MPEG2, MPEG4 ASP or AVC or something completely different for example wavelet based formats like snow or dirac), the used encoder and its algorithms and the used settings.

[Poutnik]

I don`t think there is a strict definition of compressibility. There are too much parameters involved. It depends not only on the source, but on the format (MPEG2, MPEG4 ASP or AVC or something completely different for example wavelet based formats like snow or dirac), the used encoder and its algorithms and the used settings.
I understand and completely agree. You have mentioned the opposite point of view that I have not mentioned. Of course it is true.

I was thinking in context od XviD. There is need to have general agreement
how compressibility measurement is to be done, e.g. Default settings, MPEG,
Q2.
It would has sense to get bitperpixel value at given conditions
and this value consider as measure of compressibility ( just example).

Another way/approach could be "relative compressibility" as ratio of file sizes
for e.g. Q2 and Q4.

[Doom9]

(now, worked before I think)No, it never worked. VBB's search engine is painfully inadequate.
Isn't the compressibility check just encoding samples at Q2 (maximum quality), deriving the total size when encoding at Q2 from that, then compare that to the desired size and show the whole thing as a percentage and then compare it to another movie? E.g. if you encode to 700 MB and that's 50% of the size of Q2, and then you have another movie you're encoding to 700 MB and where 700 MB is only 40% of the size at Q2, then the first movie would be more compressible because given the same target size, the size difference to perfect quality is lower. It's not a constant.. it's a ratio as you're comparing target filesize with filesize of a perfect quality rip and try to draw some conclusions from that size delta. It is reasonable to assume that if your rip has a targe size of 80% "perfect quality size", it will look better than if the target size is only 50% of the "perfect quality size".

[jsquare]

Don't know about your guys but I still have my doubts about the reliability of the Compcheck, after all it only checks portions of the source with real fast scenes changes, in not like an full analysis as on some transcorders like Shrink which checks the whole source before transcoding to the desired size/quality.

Right now I'm kind of leaning to-wards the idea of finding the average bitrate of the source and give a percentage of that rate to the encoder to work on.

Since I'm a 1-Pass encode freak(don't care much about a target size), I been trying different methods lately like fixed Q-Based encodes and fixed bitrate encodes with an 20% of the average bitrate from the source which is calculated by using ReMPEG2 on a single VOB file, and so far the results are very good but I still have to tweak some XVID settings to see which one gives the best results.

[Poutnik]

Isn't the compressibility check just encoding samples at Q2 (maximum quality), deriving the total size when encoding at Q2 from that, then compare that to the desired size and show the whole thing as a percentage and then compare it to another movie?
E.g. if you encode to 700 MB and that's 50% of the size of Q2,
and then you have another movie you're encoding to 700 MB and where 700 MB is only 40% of the size at Q2,
then the first movie would be more compressible because given the same target size, the size difference to perfect quality is lower.
It's not a constant..
it's a ratio as you're comparing target filesize with filesize of a perfect quality rip and try to draw some conclusions from that size delta.
It is reasonable to assume that if your rip has a targe size of 80% "perfect quality size", it will look better than if the target size is only 50% of the "perfect quality size".

You are right and I am proud my post is worthy enough for Doom9 attenttion :D .
I think it is compresibility check as it is used and called.
I agree it is very useful procedure and I use it myself in modified way too.
I agree it is a good measure of future quality of encoded clip.

For common reader post can ends here, for common practise it is enough.. ;)

But IMHO compressibility ( = ability of being compressed )
as a term is misused here.

As file compression analogy, compressibility is more about

"How many times is ZIP file smaller than original"

than about

"Oh no, that damned ZIP is a half bigger than my floppy, I will try RAR" :D

Such movie comparison based on TargetSize/Q2Size percentage
is applicable only to ones identical ( at least roughly )
in length, framerate and framesize ( and used CQM and little on used XviD settings )

If one use XviD exclusively for backup of standard length PAL DVD movies,
it is fully applicable. But not in general.

Such comparison should be IMHO independent on those parameters.

There are clips with very good compressibility ( soap operas),
but with bad "compressibility check" e.g. 30%
(long, large frames, small media).

OTOH there are clips with very bad compressibility (nature, sports,
noisy ones),
but with good "compressibility check" e.g. 80 %,
because they are shorter, smaller frames and I have DVD-DL :) .

These percentages talk just about how much we need
to compress a clip from Q2 state, not what is their compressibility.

If length of one movie is 120 % of another one,
it does not mean its compressibility is 20% worse.
It just means we need to compress it 20% more,
what is different thing.

It is similar as "Is 1100kbits/s high or low bitrate ?"
There is no answer. Probably low for full PAL and high for 384x288.
(It depends also on movie type, of course)

In both bitrate and compressibility topics can come
BPP (BitsPerPixel) as a measure of things.

[Kopernikus]

I don't understand why you need a defined measure of compressibility? I don't see what conclusions you could draw from that measure, if you define compressibility as a ratio between filesizes with some defined settings.

IMHO it is not practical to run 2 full Encodings in Single Pass just to find this magic number.

I think it is very difficult or even impossible to define a measure for compressibility that is independent of most or all of the parameters mentioned by you. Compressibility doesn't scale linearly with resolution or framerate.

[Teegedeck]

Testing compressibility is and always has been just a tool to determine a resolution that can be encoded at a 'sensible' range of quantizers. It is not perfectly reliable because it kind of implies that all material that is encoded at the same quantizer is at the same quality. That is not quite the case because the quality of the source material varies; not only in terms of being crisp & clean but also in how strongly the source itself already is compressed.

What you, Poutnik, are looking for seems more like a fixed ratio of the original's filesize. That's perfectly valid a point of view, too. If the original (DVD) MPEG-2 file is 3 Gigs in filesize, it doesn't make sense to keep an XviD encode of it that is 2 Gigs in filesize, one could argue. Probably the MPEG-2 file has been compressed at quite a high quantizer, too, in order to reach such small size. So it would be perfectly valid to encode it in XviD at a higher quantizer, too, and one would have a good chance that it would still look very much like the original. Provided, of course, that XviD's filesize-bloat is not only due to noise, that predator of encoding happiness! :o

But that can't be called compressiblity, and neither can what you outlined above. :)

[Poutnik]

Edit: Reply to Kopernikus

In fact, I do not need it, neither others do.
I just like things are called by their proper names
and not to like the opposite.

There is many ways how to measure compresibility.
It is just about agreement/consensus
which is used when people talk about that.
And it is not rocket science.

I have not written anything about 2 full Encodings at all.
I fact if I do some tests I use

function sample(clip c) {
cutlength=150
cuts=20
distance=int(framecount(c) / (cuts+1) )
return selectrangeevery(c,distance, cutlength, distance)
}


I agree I could mislead readers by mentioned "relative compresibility".
It is not how I would count compressibility.
It was just invented auxiliary parametr
Q2/Q4 size, only for interested, how canXviD deal with a clip.

I would count compressibility either as BPP of movie encoded
at given (and always same ) consensed XviD settings
( MV2PLAYER gives BPP in one key press )
or as a length of encode of above sample, probably normalized
acc. to frame size.

Why impossible ?
You get numbers as you get counting "compressibility chack procentage":)

I agree with your conclusions about nelinearities.
Different framerate or Different framesize give different numbers.
It is all right. Such movies, even same in content, have different properties
and different compressibility.


Just example of idea, I am not really good in CQM as Didée, Sharktooth or Soulhunter ( these guys can freely and safe correct me :D )

One can choose CQM according to how they fit to target filesize
(lets say it is related to average CQM quantizer value.

The other dimension can be according to compressibility of a clip
(lets say it could be related to CQM shape.

For high compressibility one can use "more shallow/flat" CQM
with low BottomRight/UpperLeft quantizer ratio as 6of9 has,
keeping full details.

For low compressibility one can use "more steepy?/hilly" CQM
with high BottomRight/UpperLeft quantizer ratio to better compress
residual noise/unimportant spatial/high motion details
( HVS based Sharktooth's LR/HR/UHR ones? ).

[Poutnik]

Testing compressibility is and always has been just a tool to determine a resolution that can be encoded at a 'sensible' range of quantizers. It is not perfectly reliable because it kind of implies that all material that is encoded at the same quantizer is at the same quality. That is not quite the case because the quality of the source material varies; not only in terms of being crisp & clean but also in how strongly the source itself already is compressed.

What you, Poutnik, are looking for seems more like a fixed ratio of the original's filesize. That's perfectly valid a point of view, too. If the original (DVD) MPEG-2 file is 3 Gigs in filesize, it doesn't make sense to keep an XviD encode of it that is 2 Gigs in filesize, one could argue. Probably the MPEG-2 file has been compressed at quite a high quantizer, too, in order to reach such small size. So it would be perfectly valid to encode it in XviD at a higher quantizer, too, and one would have a good chance that it would still look very much like the original. Provided, of course, that XviD's filesize-bloat is not only due to noise, that predator of encoding happiness! :o

But that can't be called compressiblity, and neither can what you outlined above. :)

I apologize ( twise for mods :o ) , If I confused you.
I fully agree with your first paragraph.

Something I hope I made clear in my previous post.

I think either BPP or
( length of 3000frames*720*576 PAL movie) at XviD defaults
could be considered as good compressibility measure.

BTW, Twice as proud, being answered by two Doom VIPs :D

In fact I do not work with DVD at all,
I process usually 384*288 or 480*576 analog TV PAL captures.