Xvid and the Question of Quality!

[dihelson]

Hello, fellows,

Once in a while, we see people here questioning about the way we should set Xvid Codec and other software in order to get the best possible quality.
Then, every time, then comes a "little" guy telling us not to asking anything about the quality, because "Quality is a personal taste".

Oh, boy...
Well, I'm very curious about this kind of answer and philosofical posture. See, digital image is a mathematical thing. We can measure it. We're not speaking about Psicology! So, I think he have means to measure the quality of a picture, don't we ?
I think it's not so subjective...
If quality is a subjective thing, then we are all subjected to the developers "taste" of quality. So, how the developer know that he is improving the Codec ? If not a measured thing ? Just looking ?
Is it eye measured ???
No, no!
I think there's methods to measure the quality.
IMHO, The quality tends to be better if the final result can be close to the Source, and this is obvious. If the final Xvid seems like the original DVD, then it has high quality. Nobody discuss here the DVD quality, so, the more close to DVD, the better quality.
These things sound very primary, I know, but I've seen that although almost everybody agrees with it, some people continues to think that the quality can't be measured. How not ?
We're not talking about Angels sex. We talk about a mathematical model.


Regards,
Dihelson

[Koepi]

The "question" of yours got lost somewhere in your post, and I don't exactly understand what your problem is ;)

If you'd look around, there are plenty of PSNR and other mathematical comparisons floating around in this forum. All mathematical measures for quality.
One flaw on the other hand: you can't say because XviD has a PSNR on that particular test of 50 and DivX has 48, that XviD is superiour.

Let me explain it this way: sysKin invented/implemented a thing called TOO_SMALL_LIMIT. It was an attempt to reduce bitrate without loosing quality, and it was successfull: PSNR increased!
But wait. Watching the video, you saw a bad side-effect: the video got unsharp and a bit blurry.

So these mathematical models can help you to improve things, but you always have to verify if it really helps the quality.

I hope this helps,

Regards
Koepi

[U977]

When I read your post, I was tempted to answer something similar to what Koepi said.

If you look at the current codecs comarisons, you will see that, as example, the current DivX beta versions (french polynesia: Kauehi, manihi,...) and the latest XviD versions have a very very close PSNR.

However, DivX always gave a slightly less detailed movie, while XviD retains more details. See Doom's codec comparison, as example again.

RV9 codec is definively not the codec I would use for movies, because the encode looses something that gives life to the movie.
Yet, the movie is still of good quality: no blocks, no mosquito noise,...
The images are not the same, as when you apply filters to increase compressibility. This is why I would also say that quality is also a matter of taste.

[Sagittaire]

PSNR is mathematical test but SSIM and VQM are Psycho-visual mathematical tests and the RV9 or WMV9 are the best in this test ...

My Eye's test say too RV9 EHQ and WMV9 with no PP are better than XviD and DivX with details or blocks ...

XviD and DivX are codecs Iso-MPEG4 which cannot fight against codecs which technologically seem more advanced ...

[MfA]

Originally posted by dihelson
I think it's not so subjective...

That is not so much thinking, as feeling ... once you start thinking you will see it our way :)

If quality is a subjective thing, then we are all subjected to the developers "taste" of quality.

No matter how much the taste differs between two people, the taste between any given objective quality measure and a person will on average almost certainly differ more.

So this is not such a bad thing, the alternative (totally relying on an arbitrary objective measure) is almost always worse.

IMHO, The quality tends to be better if the final result can be close to the Source, and this is obvious.

How do you measure if something is close to something else? :)

You might want to read up on quality measurement of audio, audio has the nice property that phase is nearly never an important component ... as such two sounds which have a huge distance in an euclidian sense can sound almost the same, and sounds with a much smaller euclidian distance can sound completely different.

Even though this is more pronounced in audio you can see the same for images. To see how badly MSE (and by extension PSNR) predicts quality take a look at the demo images here (http://www.cns.nyu.edu/~zwang/files/research/quality_index/demo.html). This is not a matter of personal taste. Everyone who isnt blind would agree that the blurred and JPEG compressed versions of the images look far worse than the same images with a mean shift or when contrast stretched, even though they have the same euclidian distance.

Such cases can be found for any objective measure, in the end all objective measures are designed to approach the results of subjective measures ... and they all fall short, in fact they fall short to such an extent that a poor subjective measure with only a single person eyeballing it is usually better!

[Didée]

Let me give another analogy.

There was an extensive scientific research about "what makes a beautiful face". (I've once read a lengthy article in the German magazine "P.M.", and also stumbled over that elswhere I can't remember actually.)

The scientists made big efforts with a large field research, asking several thousand people about dozens or hundreds of different faces. Then, they broke down human face into dozens of different metrics, and modelled some kind of a formula out of that.
Now, in general, that worked. Faces that were commonly seen as more beautiful mostly obtained a higher metric value, and the less beautiful faces a lower metric.
But then they tried to model the perfect face ... a virtual constructed face, with a much higher metric than any of the real faces in the research. Guess what? This "perfect face" did look O.K., perhaps even good. But it looked -

- BORING TO DEATH.

(And some of you may know, regarding video compression, that's exactly my impression of *PEEEEEEP*)

- Didée

[ultimatebilly]

In a book about video-compression I have, there is a nice example to illustrate what flaws PSNR-tests have...
There is a picture with a girl who is smiling, so that you see the front-row of her teeth.
There are two versions of the image. The first one was compressed a bit too hard, so there are compression artifacts and quite a lot of mosquito noise, but it still looks ok.
The second picture doesn't have any compression artifacts, except for one (worst case scenario).
The value for one macroblock has gotten lost, so it is entirely black - this block was in the middle of her teeth...
So, which picture do you think had the higher PSNR? The one without the mosquito-noise of course - but if I had to choose, I would take the girl without the black tooth...
PSNR tests don't care about irrelevant information - for example blocks in areas which are extremely dark. A good codec will throw some of the information there away to save bits, because you won't see it anyway, just in the way a good codec will save bits in extremely high-motion scenes - but these frames will receive a low PSNR-value, even if they look perfect to your eyes...

[TheXung]

Originally posted by dihelson

IMHO, The quality tends to be better if the final result can be close to the Source, and this is obvious.

Not if the source is bad, now this is obvious.

The science behind video compression is designed to model natural images and that coincides with our desire to watch natural images, not noisy or distorted images. Quality and closeness to the source are not the same thing.

[MfA]

No reference objective quality measurements are even harder, the assumption that the source is very high quality and can be used as a reference makes things a little easier.

[Peters]

Originally posted by Sagittaire
PSNR is mathematical test but SSIM and VQM are Psycho-visual mathematical tests and the RV9 or WMV9 are the best in this test ...

My Eye's test say too RV9 EHQ and WMV9 with no PP are better than XviD and DivX with details or blocks ...

XviD and DivX are codecs Iso-MPEG4 which cannot fight against codecs which technologically seem more advanced ...

My eyes say that RV9 looks better at low bitrates only because it removes much details than Xvid or Divx

[Sagittaire]

If XviD were more detailed than RV9 EHQ or WMV9 the PSNR, the VHQ and the SSIM would be higher for XviD ...

When video is macroblocked it will have me to be explained how one can see the details ...

RV9 EHQ is everywhere the best in for weak bitrates and WMV9 is best for the tops bitrates ...

[temporance]

Sagittaire,

You can't argue that because [insert quality metric here] says that one codec is better than another, then it must be right.

The golden test of how well a codec performs is a pair of eyes. And each pair of eyes might give a slightly different result. Eyes that are very experienced in video compression are more sensitized to blocking artefacts than the average pair of eyes.

A few block edges appearing in a very textured region don't do much harm. If these blocks are allowed in order to save some bits for fine skin textures, for example, then we get a better looking picture albeit with some hidden blocking.

When a codec is over-cautious and tries to avoid all blocking, then, IMHO, it will often smooth over some of the real fine details. And by doing such filtering in loop, it can get a better result in the quality metrics. A person who hates blocking will prefer a codec like this, but I agree with Peters, a small amount of hidden blocking or ringing is a price worth paying to avoid a smooth, artificial looking image.

EDIT: the performance of RV9 is down to personal preference, but I'm sure of one thing: WMV9 is definitely not better at high bitrates - in my tests it performs significantly worse than the MPEG-4 codecs at higher bpp ratios.

[Peters]

Originally posted by Sagittaire
If XviD were more detailed than RV9 EHQ or WMV9 the PSNR, the VHQ and the SSIM would be higher for XviD ...

When video is macroblocked it will have me to be explained how one can see the details ...

RV9 EHQ is everywhere the best in for weak bitrates and WMV9 is best for the tops bitrates ...

Funny because i was speaking of what my eyes said and you talk of
mathematical results :)

PSNR, VHQ and SSIM do mathematical and global results.
And yes a frame can be much blurry and have a greater PSNR than an other.
And all these tools are global so you can have an area of the frame with lack of details and the rest whith good results.

I have seen your test(LOTR2, Harry Potter) and seen your results.

Yes, RV9 is very pleasant but if you don't see there is much details lost(see the mountains!), then i suggest you to verify your eyes.

You say RV9 EHQ is the best for weak bitrates and i agree , the most of the time.RV9 looks the more pleasant for the eyes but..

..but RV9 is also, for me, the codec who respects the less the details of the original.

By the way, i can give you a sample of a movie (a western) with very slow action(men walking with vertical movements of the camera) where RV9 is awful.... and not Xvid or Divx


Edit: just a trivial example of the lack of details with RV9.
Your trailer of LOTR2 at the 39th second.
See the mountain beetween the two men, wow..

[Lefungus]

IMO, these tools like psnr, vqm, ssim are useful when you try to improve one codec performance, but not really for codecs comparisons.
Their worst point is that they don't take motion into account. So they're not really adapted to be video quality metrics yet.

[Sagittaire]

And yes a frame can be much blurry and have a greater PSNR than an other.

Perhaps with the PSNR but not with the VQM and the SSIM. The VQM and the SSIM are test to develop by American professors specialists of the image processing ...

in my tests it performs significantly worse than the MPEG-4 codecs at higher bpp ratios.

In the very high bitrates visually all the codecs are worth and the only means of differentiating them it's a test (PSNR ...) and these test say WMV9 winner ... but all the codec are very good in very high bitrate: the DivX3 SBC too for example ...

I know and I have to test all the codecs existing (XviD Devapi3 and Devapi4, DivX beta, WMV9, RV9 with better setting ...) with many ripp and my eyes say that for a compressibility lower than 50% compared to XviD in as 2 RV9 EHQ is the best and that for a compressibility superior to 50% the WMV9 is the best ...

For a lower compressibility 40% the Iso-MPEG4 codecs are really HS (Blocking and ringing ...) whereas the WMV9 and especially RV9 EHQ still have a remarkable quality ... make the test ...

In conclusion me I am not a specialist of the image processing and you smell yourselves more qualified than of the professors of university of which it is the trade ... ;-)

[Peters]

Originally posted by Sagittaire

I know and I have to test all the codecs existing (XviD Devapi3 and Devapi4, DivX beta, WMV9, RV9 with better setting ...) with many ripp and my eyes say that for a compressibility lower than 50% compared to XviD in as 2 RV9 EHQ is the best and that for a compressibility superior to 50% the WMV9 is the best ...


blablabla...

instead of being sarcastic tell me what your eyes think of the details of the mountain in YOUR RV9 ENCODING of LOTR (and i can give you many other examples)

You prefer to show your results of PSNR, VQM and SSIM but for me it prooves that RV9 is the best for PSNR, VQM and SSIM scoring, that's all.

With RV9 details are going away and i don't need any tool, just my eyes, to see it

Have a nice day...

[Sagittaire]

What think you of the blocking/ringing of XviD on each change of scenes and in all the fast scenes ... The LOTR II with 570 and 715 Kbps is very bad than RV9 EHQ or WMV9 version ... I post them if you want ...

I does'nt understand how you can say that XviD is better than RV9 EHQ or WMV9 on the trailer cards than I posted ... Buy glasses ...

I think that if I posted the source and the encoding in XviD you would still find the means of saying that XviD is better ...

I think that one can discuss by making a comparison WMV9/RV9 but XviD is clearly worse ...

that do not interest me to discuss with somebody who is not objective ...

[temporance]

...you smell yourselves more qualified than of the professors of university of which it is the trade...Several people here including me are lucky enough to have video compression and quality as their trade. The nice things about this forum is that university professors appear behind a nickname like the rest of us! :)

that do not interest me to discuss with somebody who is not objectiveBut video quality is inherently subjective! Just because someone disagrees with you doesn't mean either of you is wrong. The only conclusive way to find out which is better - MPEG-4 or RV9 is to do a scientific survey with a large number of non-expert viewers.

Today's objective metrics (VQM, JND, etc) are far from perfect and, in tests, only perform slightly better than PSNR! Lefungus has a good point, metrics are most useful for detecting small quality improvements working on optimizing one codec, and not so good for comparing radically different video compression technologies.

I'd rather trust my own eyes!

[MfA]

Even if there are a couple of metrics which take motion into account, there are no metrics I know of at the moment takes into account the persistence of artifacts in a given location or along a given motion trail. Which is a big factor in how annoying they are.

All metrics have some stories about the underlying physical model and how it is close to human vision ... but in the end it is all BS, the only thing they care about is if it correlates well with subjective testing. For instance the assumptions underlying SSIM and VQM are radically different. SSIM assumes our eyes are mainly sensitive to spatial structure, VQM assumes our eyes are mainly sensitive to spatial channels (a local frequency space). Which is right? Noone cares, which gets closer to subjective testing is all anyone really cares about (read the papers).

[Peters]

Originally posted by Sagittaire
What think you of the blocking/ringing of XviD on each change of scenes and in all the fast scenes ... The LOTR II with 570 and 715 Kbps is very bad than RV9 EHQ or WMV9 version ... I post them if you want ...

I does'nt understand how you can say that XviD is better than RV9 EHQ or WMV9 on the trailer cards than I posted ... Buy glasses ...

I think that if I posted the source and the encoding in XviD you would still find the means of saying that XviD is better ...

I think that one can discuss by making a comparison WMV9/RV9 but XviD is clearly worse ...

that do not interest me to discuss with somebody who is not objective ...

I suggest

1) To read exactly what i wrote, i never said XVID is better than RV9 EHQ , i said RV9 is the MOST PLEASANT CODEC for low bitrate but yes, we loose more details with RV9 than with XVID.
That's all.

2) Change your eyes if you don't see the lack of details in your RV9 trailer (remember i suggest you to see the 39th second )
Do you want i put screenshots ?

3) I think i'm much objective than you are . By the way, my father learned me to always tell "I like this or i diskike it" instead of "this is good or this is bad"

[Sagittaire]

download and compare ...

http://jfl1974.free.fr/Video/Kaukura-570.rar
http://jfl1974.free.fr/Video/RV9-570.rar
http://jfl1974.free.fr/Video/WMV9-570.rar
http://jfl1974.free.fr/Video/XviD-570.rar

RV9 EHQ and the WMV9 cannot be the best in 100% of the frames ... but visually with low bitrates there is no possible comparison. XviD and DivX (There is Kaukuru 3 pass slowest encoding) is really bad ...

Generally RV9 EHQ and WMV9 are more detailed than XviD with all bitrate ...

For encodings has less than 40% of compressibility the iso-MPEG4 cannot fight ...

tests PSNR, SSIM and VQM confirm what the eyes see ... ;-)