Objective codecs comparison

[Gabriel_Bouvigne]

For those of you who read french:

http://forum.hardware.fr/forum2.php3?post=31251&cat=3&config=&interface=0&cache=cache&p=1&sondage=&owntopic=&trash=

I'll post explanations in english later...

[Koepi]

Why don't you wait until start of the next week?

I think there will be some explanations necessary then why those tests are clearly flawed ... ;)

[Gabriel_Bouvigne]

Like the fact the you can't completely judge a psychovisual codec using objective criterions?
Like the fact that it's a video codec and not a still picture codec?

Yes, probably a lot of things...

See you next week for debate.

[Koepi]

Hum? Was there something negative in my post?

i thought smileys speak for themselfes...

*shrug*
Koepi

[Gabriel_Bouvigne]

It's not negative. A good debate is often in interesting thing.

And yes, there are some points to point.

[avih]

here's a link to an automatic translation by google to the above comparision:


http://translate.google.com/translate?sourceid=navclient&hl=en&u=http%3A%2F%2Fforum%2Ehardware%2Efr%2Fforum2%2Ephp3%3Fpost%3D31251%26cat%3D3%26config%3D%26interface%3D0%26cache%3Dcache%26p%3D1%26sondage%3D%26owntopic%3D%26trash%3D

grrr, just tested. the link doesn't work. it does, however, leads you to google translations page, continue from there

[AM Tiainen]

You can view MPEG-4 codecs comparisor from http://www.nixed-hq.com and press "Codec Comparisor" link
or directly http://koti.mbnet.fi/website/mpeg4test/

Also http://mukoli.free.fr/video/ has some codecs comparisor DivX 4 vs DivX 5.

[morello12]

Wow, both those reviews think prettly badly of Xvid. They rank it both below DivX 4, and the second review says it's not very good at all, and gets incredibly blockly.

Well, IMHO, codecs can only go so far. There may very well be just as many problems with Xvid as there are with the people testing it ;)

It's almost as bad as when people test aac (or some other format, like ogg) vs mp3 and then use a 3 year old Xing mp3 encoder or something stupid like that in their tests. Or even worse they don't even tell you, so their entire tests are meaningless to me.

[soujir0u]

Eh? I thought that koti site said XviD was better than DivX4? Didn't bother checking the other sites.

[Gabriel_Bouvigne]

Nothing about objective comparisons?

[MaTTeR]

I really don't understand how all these comparison tests are saying that Divx 5 is sharper than Xvid. I mean in all the comparisons I've done on my own machine, it was clearly Xvid that held better detail in all areas. I accomplish this by using the defaults in Xvid. LOL

I must say that I also don't have any issues with blocks except on B&W source using Xvid. Every codec I've tried fails miserably on B&W sources.

[-h]

Nothing about objective comparisons?

What kind of test was it?

I haven't been keeping up :eek:

-h

[Gabriel_Bouvigne]

It's not a test that was made. It is a test that you have to make. (see the first post)
I'm giving you tools to run objective tests.

[-h]

Hm the only objective test I know of is file size per fixed quantizer, which DivX5 cheats on anyway (B-frames are always encoded with the quantizer doubled, even when the quantizer is restricted). I guess you could develop a perceptual SNR comparison, but that wouldn't be much fun, and would run into the same issues that EAQUAL found.

Ah this looks interesting, I just got the file.

-h

[ReferenceDivx]

I really liked the layout of your tests. It was clear and concise. It might not show the whole picture about the different codecs, but it does show some strengths and weaknesses of every codec.

If i had a choice between paying for a codec and not, i would go for not paying for a codec. - Xvid rules!

[robUx4]

I posted one of these negative reviews about blocky XviD.

But I didn't know you have to enable post processing to remove that block effect.

Since the DirectShow filter doesn't have it enabled by default, I just used what I had "out of the box".

If I find some time (very unlikely) I'll post a new test with also a source with a bit better quality.

From a fast test it seems that the quality of XviD is much better than DivX5. But there is a fast motion problem (you can download 2 AVIs at the end of my review to see the difference http://mukoli.free.fr/video/ )

[AM Tiainen]

Originally posted by ReferenceDivx
If i had a choice between paying for a codec and not, i would go for not paying for a codec. - Xvid rules!

If i had a choice between paying for a codec and not, i would go for not paying for a codec. - Xvid rules!

It is very hard to rate DivX 4 and XviD and that which is better. Because XviD picture is not as sharp as DivX on lowmotion but on faster motion XviD is better and in some pictures case is different. But that i agree that XviD is very good codecs.

We don't want rate codecs by that are they cost or not. Example some people say that in Linux is better only because that is free, unlike Windows. If want free codecs they XviD is good choose. But if look support and feature, Windows is pretty good althought it cost. But in test we don't want think is codecs is cost or free. And DivX 5 Pro can download with Ad-ware if want free version and if can accept that you DivX Pro Ad-ware show 4-5 ad per week when surfing on the net.

[Nic]

Its good to see someone so illustrious in the LAME/audio community help test XviD.....maybe we could rope ff123 to do some real statistical tests :D

Cheers,
-Nic

[Gabriel_Bouvigne]

I've just uploaded a new release

[Gabriel_Bouvigne]

I've uploaded a new release on http://gabriel.mp3-tech.org

[-h]

Hi Gabriel,

Sorry I haven't been able to play with your program yet - I will make time this weekend.

What have your findings been so far?

-h

[Ookami]

Originally posted by Gabriel_Bouvigne
Like the fact the you can't completely judge a psychovisual codec using objective criterions?
Like the fact that it's a video codec and not a still picture codec?

Yes, probably a lot of things...

See you next week for debate.

Ooooops, silly me. I should READ the threads in that I post. "/%#) maybe if I didn't had this URLMON crashes all the time I could saves all this nice threads and read them offline. Online, I never take the time to read it all properly.

Anyway, I've already wrote this post offline, so I will post it just to show how dumb I am.

Enjoy (?):

I disagree with anyone that claims that image comparing tools can be used for "objective" codec comparisons.

IMNSHO, Codecs should be compared for what they are made (videos, no pictures)... Like Nando, once wrote, all those "let's take some pictures and compare them" codec comparison are not what they should be. As far as I know only Doom9 does it right, altough even his test isn't near objective testing. There are quite a few "codec comparisons" floating around and many of them don't even specify the exact settings, how many people were the testers etc. Real people should be the one who are comparing the codecs, not some automatized process.

Even if fu2k would unleash his "custom algorithm that aims to model how the human eye perceives image distortion" to compare between different codecs I do not think that it could be called objective testing.

If someone could emulate perfectly, the human perception of moving pictures (with adjustable settings of course ;-) ), that would be another story...

The same with audio... If I want to compare audio codecs I will certainly not compare tones (right word?), but whole audio files. Altough, I'm one of those weirdos who don't think that ABX testing is objective... *grin*

As you all have noticed I don't like the word objective :D . Probably, because I don't think that humans can be non-biased.

As an end note, this was not meant as a personal attack or putting down your efford (and software), but to show a point of view, that is somewhat missing in many of those "codec comparison" discussions.

If I'm missing something, or am completly wrong, please enlight me. Thanks.

All the best,

Mijo.

Originally posted by Gabriel_Bouvigne
Nothing about objective comparisons? "

[Gabriel_Bouvigne]

Objective doesn't means at all that it will tell you wich one is best for a specific situation.

Objective means that the measurement is not based at all on human opinions, and is 100% reproducible. So my program gives you objective results.

Of course objective difference is not the only point for a psychovisual codec. But it allows you to focus easily on certain parts while testing and developping. (btw this subject is also spoken about in the readme file shipped with my prog).

I totally agree that the final judge for an mpeg codec should be a set of humans. However for development, there could be other needs.

[Ookami]

Hello,

thank you for your nice reply.

-specific situation

For what purpose is the test started? Why call it test at all if you don't test anything ;-) ? Just joking. I see your point...

-objective

We could talk forever about objective or not. But I simply don't like the word. Scientific, yes. Objective...

-development

I agree. And it seems that we agree.

Like I stated above, if I had read your postings more carefully I wouldn't post in this thread at all.

All the best,

Mijo.

Originally posted by Gabriel_Bouvigne
Objective doesn't means at all that it will tell you wich one is best for a specific situation.

Objective means that the measurement is not based at all on human opinions, and is 100% reproducible. So my program gives you objective results.

Of course objective difference is not the only point for a psychovisual codec. But it allows you to focus easily on certain parts while testing and developping. (btw this subject is also spoken about in the readme file shipped with my prog).

I totally agree that the final judge for an mpeg codec should be a set of humans. However for development, there could be other needs.