Doom's latest codec comparison

[Gazza]

Congratulations to xvid's developers and supporters for taking out the line honours in Doom's latest comparison tests. A great result and well deserved praise to you all.

[kilg0r3]

I'd really like to know how rv9 would have scored with anamorphic encoding. Anyway looking at the results of doom9's test and those of the ct comparison a big cheer to the xvid team.

On a personal note, I really hope that Nic rejoin us soon. I am sure to miss his warm and clear postings.

[tiki4]

Thanks to all the effort taken as well from me. Sorry, I tried to find out myself, but what happened with Nic? Did he leave in anger or just because of lack of time? I didn't see any 'farewell post' like that from DSPGuru around here.

tiki4

[bilu]

If you look at Nic's profile:

http://forum.doom9.org/member.php?s=&action=getinfo&userid=519

you'll see that last message is not this one:

http://forum.doom9.org/showthread.php?s=&postid=305304#post305304


Which by itself doesn't mean anything, but it seems he's still motivated :)


Bilu

[Doom9]

kilg0r3: it doesn't matter, because to be fair I'd have used a lower resolution for the other codecs as well and made my player stretch the video upon playback. Using a non 1:1 PAR only results in a higher bit/pixel ratio at a given playback resolution.

[Sirber]

The codec comparison would have been better whitout post-processing. PP remove details...

@Doom9

No flaming :)

[Doom9]

and I bet you never use postprocessing during movie playback ;P I couldn't care less how something looks that I never get to see.. The majority of the people reading that comparison use postprocessing during playback of their ripped movies. Therefore, I reviewed the movies the way people (including myself) will watch them. If you're not happy do your own comparison.

[bilu]

Eventually that could lead to a HUGE encoder and decoder comparison... :rolleyes:

[manono]

Hi-

I bet you never use postprocessing during movie playback

I, for one, don't use any. If you do it right in the first place, you don't need postprocessing. But even if you do ordinarily use PP for playback, when trying to determine the quality of the encode, I think it should be turned off. It's a minor point, and doesn't take anything away from the excellent job you did with the comparison, but I agree with Sirber.

I couldn't care less how something looks that I never get to see

and, from the comparison, at the bottom of the Matrix page:

Personally I value details a lot

I'm not quite sure how you can reconcile those completely contradictory viewpoints.

[AS]

I think what Doom9 meant was if he can see enough details as it is with post-processing, then you can be certain it has more details without post-processing.

[Angrychair]

Originally posted by AS
I think what Doom9 meant was if he can see enough details as it is with post-processing, then you can be certain it has more details without post-processing.

Maybe I'm alone in this opinion, but you shouldn't try to put words into other people's mouths.

I'll have to agree, using post filtering on an encoding quality comparison is at best questionable, and more likely foolish and short sighted.

Just because YOU only watch your videos with PP enabled doesn't mean the comparisions should be made that way, especially if a different decoder/postprocessor is used for each codec.

I leave PP off when I watch videos, and the ones that I encode myself look great that way, too bad everyone doesn't follow my lead, the mpeg4 encoding world would be better off that way.

[Assault]

Keep in mind that you can't deactivate RV9's post-processing. Because of that it wouldn't be fair to deactivate it for the other codecs.

Regards
Assault

[masken]

It was a good and informative comparison. Thanks :)

[Doom9]

hehe.. assault just made a good point I wanted to make.
And, I did watch some clips without PP but in the end I like the output with postprocessing more. Keep in mind that most likely your video setup is inferior to mine.. you already have a natural filter when you have unnecessary a/d and d/a conversions in your display chain. And all the VCD/SVCD/DVD-R folks have a really strong filter called TV.

Bottom line is that I like the postprocessed output better than the non postprocessed... and I also didn't use full postprocessing strength but a compromise in between leaving details and improving the general harmony of the picture. If some blocks disappear when I do PP and the image is still sharp and has details in faces and such, then I cannot object to that. If PP takes away all details (I tried XviD with activated postprocessing and ffdshow PP on top) only then that's a nogo.

I'll have to agree, using post filtering on an encoding quality comparison is at best questionable, and more likely foolish and short sighted.
and you're an ungrateful bast*** who just got two very well deserved strikes.

[sungey]

hmm if RV9 post processing cant be turned off .. i guess making comparison with post-processing turned on is needed .... its foolish to compare codec unfairly ... (RV9 with post-processing and no PP for the others)

[mf]

Originally posted by sungey
hmm if RV9 post processing cant be turned off .. i guess making comparison with post-processing turned on is needed .... its foolish to compare codec unfairly ... (RV9 with post-processing and no PP for the others)
Dude! I was going to say that!

[Angrychair]

Originally posted by Doom9
hehe.. assault just made a good point I wanted to make.
And, I did watch some clips without PP but in the end I like the output with postprocessing more. Keep in mind that most likely your video setup is inferior to mine.. you already have a natural filter when you have unnecessary a/d and d/a conversions in your display chain. And all the VCD/SVCD/DVD-R folks have a really strong filter called TV.

Bottom line is that I like the postprocessed output better than the non postprocessed... and I also didn't use full postprocessing strength but a compromise in between leaving details and improving the general harmony of the picture. If some blocks disappear when I do PP and the image is still sharp and has details in faces and such, then I cannot object to that. If PP takes away all details (I tried XviD with activated postprocessing and ffdshow PP on top) only then that's a nogo.

and you're an ungrateful bast*** who just got two very well deserved strikes.

Sticks and stones, my thin-skinned friend, sticks and stones. How about striking yourself for that 'bast***' comment?

What you like better isn't really relevant and in fact really throws the entire result into question. You're now letting your opinion skew the entire test.

The fact that RV9 doesn't allow you to disable post processing would probably better be served with a note to that effect, rather than enabling 8 other post processing schemes that serve to mask what the codec is actually producing.

Any way, feel free to ban me for 30 days, or whatever makes you feel like a Big Man™, you can silence me, but you can't change my opinion through such a vulgar abuse of moderation powers.

[kilg0r3]

whew, it is hot in here!

[DaveEL]

Originally posted by Angrychair
How about striking yourself for that 'bast***' comment?
Whats wrong with that no forum rules are broken.

What you like better isn't really relevant and in fact really throws the entire result into question. You're now letting your opinion skew the entire test.

Yes is it. These type of tests are ALL opinion. When i evaluate codecs i use the setting when make them look best to me, Thats the way i will watch them so that the MOST valid comparison IMO.

The fact that RV9 doesn't allow you to disable post processing would probably better be served with a note to that effect, rather than enabling 8 other post processing schemes that serve to mask what the codec is actually producing.

Well for me this is a non issue you examine the output you have as you would watch it. If one has PP and one doesn't it doesn't matter. If i encode with rv9 i have to have PP so the test has to reflect that if i think the other codecs look better without i have that option so it fair to compare with and without pp IMO.

Any way, feel free to ban me for 30 days, or whatever makes you feel like a Big Man™, you can silence me,

Well the first strikes may have been a little harsh IMO (but doom9 is the law around here so its his option to strike you), but this definitely deserves a strike so goodbye.

but you can't change my opinion through such a vulgar abuse of moderation powers.
Thank you for that comment, you prove my first point, it is OPINION that matters.

DaveEL

[Sirber]

Originally posted by Assault
Keep in mind that you can't deactivate RV9's post-processing. Because of that it wouldn't be fair to deactivate it for the other codecs.

Regards
Assault

You can see squares in RV9 encodes too, even if it's post-processed. I'll ask Karl if he can add an option to disable it the next player release.

[((( atom )))]

@doom9

hmm, i would have really appreciated to see you also compare the codecs at a higher bitrate. personally, i have a vga-tv and 1cd-rips are a nogo for me. your comparison actually tested how good a codec can compress but it would have been nice to also see how good a codec can really perform on the other hand. also people start to use mpeg4 in a production environment, and there people care about the maximun possible quality a lot.

also when testing at higher bitrates i suppose that another very interessting effect would have come up: at what filesize do the codecs saturate? imho that is a very interessting point, since they do reach their absolutly unincreasable maximum-quality there and it would show what the different codecs are really capable of.

higher bitrates also bring live into the post-processing discussion, since imho a 2 cd-rip with a proper codec looses quality again, when being post-processed.

i hope you take this as constructive criticism and don't tell me to go and make my own comparison. people read your reviews, not mine and personally i am perfectly happy using xvid.

and last not least, thx for putting all that work into this comparison again.

[Assault]

@ Doom9

I finally read your long comparison and I don't think that you chose neither the wrong bitrate nor wrong playback settings. ;)
Thank you very much for putting so much free time in it.

@ ((( atom )))

You perhaps didn't mean it this way but I don't think that we should demand from Doom9 to do another comparison with different bitrates because it's a lot of work to do such codec tests. (I know that because I began one by myself not long ago ;)) Doom9 is right here. If we want different settings we should do our own comparison.

Regards
Assault

[((( atom )))]

@ assault

i do not demand anything. if someone goes and puts a lot of work into something, wich i really appreciate, he will surely be interested in peoples reactions about his work. i would be. further: why would anybody use this forum if not for the exchange of opinions? well, he /raised/ this forum /as/ a forum, not a digest and i feel free to put my opinion into a discussion. no complaints and no demanding.

[crusty]

OK, lots of quotes, sorry if that bothers you, but bear with me for a minute. And I'm not trying to be flamebait. :D

Sirber:
The codec comparison would have been better whitout post-processing.

Doom9:
and I bet you never use postprocessing during movie playback ;P I couldn't care less how something looks that I never get to see.. The majority of the people reading that comparison use postprocessing during playback of their ripped movies. Therefore, I reviewed the movies the way people (including myself) will watch them.

Well that's partly true Doom9, but testing like this is always subjective. By reviewing the codecs with PP turned on, you reviewed an already subjective test with another uncertain factor. Offcourse it's your choice and I'm not saying it makes your comparison null and void. Everybody in the know knows you've put a lot of work in it.

Manono:
I, for one, don't use any. If you do it right in the first place, you don't need postprocessing.
I agree. It's the proof of craftmanship to make a rip that needs no postprocessing to look good. It will also look good on much slower computers. I think it's no secret that PP needs a lot of cpu power.

But even if you do ordinarily use PP for playback, when trying to determine the quality of the encode, I think it should be turned off.
I also turn it off. Especially blockiness will show up quickly that way.

As:
I think what Doom9 meant was if he can see enough details as it is with post-processing, then you can be certain it has more details without post-processing.
Well I don't know if Doom9 meant that, I can't see in his mind (don't know if I would want too :D), but you right. If the rip has the same amount of detail as the original with a certain level of PP, it's no problem to turn it on.

Angrychair:
Just because YOU only watch your videos with PP enabled doesn't mean the comparisions should be made that way, especially if a different decoder/postprocessor is used for each codec.
Well, it's HIS comparison, and he put a lot of work in it. So he can do whatever the **** he wants with it. He could encode a clip with invisible flying monkeys in it for all he cares. He did the work.
But you've got a point about the different PP in each codec.
PP implementations are different for every codec, which would mean that some codecs will use more processing power than others.
This could mean for instance that an SBC rip would play happily at the same level of subjective quality on a 800 MHz pentium III, while the same rip done with Xvid or RV9 could need PP that would make even the fastest Pentium III choke.
While this was not a factor in Doom9's test, it could be important to people trying to make a good rip.
Also one codecs implementation could be of lesser visual quality or buggier than the others. This would also add another uncertainty to the test.

Assault:
Keep in mind that you can't deactivate RV9's post-processing. Because of that it wouldn't be fair to deactivate it for the other codecs
Well you could also consider the RV9 codec PP as a form of 'cheating'.
I guess Doom9 turned PP on the others on to prevent a comparison biased towards RV9.
Personally I think RV9 should either have been disqualified, like mpegable X4 and VP5.

Doom9:
And all the VCD/SVCD/DVD-R folks have a really strong filter called TV
TV let's you get away with just about anything. It brings tears into my eyes to see some of those encodes.....
For them ANY codec comparison would be pretty pointless I guess, at least with differences as small as the ones in this comparison.
Basically all we can tell those people now is: Don't use 3ivx.

I also didn't use full postprocessing strength but a compromise in between leaving details and improving the general harmony of the picture.
Basically you picked the settings that gave the overall better impression. I can dig that.

Angrychair:What you like better isn't really relevant
You're now letting your opinion skew the entire test.
Like I said, it's HIS work. He's blooming well right to strike you for being such an ungrateful bas****. And it's HIS opinion a lot of people value. That's why he puts it up on the site for people to read it in the first place. You can disagree with the settings he chose, but saying his opinion is irrelevant is downright cheeky.

DaveEl:
Yes is it. These type of tests are ALL opinion. :D
You said it m8!
Unless we get some sort of program which uses a reversed-psychovisual model to asses the quality of an encode based on what people are actually supposed to SEE, any comparison like this will be subjective.

Atom:at what filesize do the codecs saturate?
That IS indeed an interesting point. Especially since we haven't seen many codec comparisons explore that part of the picture.
And Atom, as you can see, the SPR test was a 2CD Rip.

Doom9 actually was very clever with the adding of the Futurama test, because animation is notoriously difficult material, and it is also episode-sized, which people do a lot as well.

@all:
Please try to stay polite and constructive please, getting into flamewars is the #1 reason forums go sour.

[Doom9]

you know.. I go by the general standards for these comparisons.. most people live by the "squeeze my movie onto one disc" mantra.. after a certain point this is no longer valid so spr was put onto 2 CDs. I'm not going to chance the testing method in the forseeable future. There are a gazillion of different options and everybody likes what he does best.. the point is picking a common low denominator and use it.. and that's what I did. I'm not going to drop postprocessing or the # CD fixation in the future, you can take that for pretty much granted. And from personal experience I know that minorities are often the loudest group out there. There's a large number of people who are happy with the comparison as it is and you can never satisfy everyone. I believe that I've picked good settings and the amount of people coming to my site to read that comparison must mean I'm doing something right.

[jeremymacmull]

@doom9 (and others)

To be fair i thought the codec comparison was very enlightening
as i always thought that now with 5.05 that SBC would be out the window but its still a very valid choice.

Only one question and sorry if you have mentioned it. How many passes did u do using divx 5.05 on SPR and matrix and in your opinion did you notice (if used) a difference between divx 5.02 in your last codec test and 5.05 multipass

if these are hard to answer and u would prefer not to thats fine but ive been trying to follow the divx 5.05 multipass vs 5.02 2 pass as to which is better and im not really getting anywhere. (i have not really got the time to do exhaustive codec comparisons and ive got a slow machine) just to clarify im not a purist i doubt id use xvid its too complex i use gknot and time encoding matters a lot as ive got an 850 p3 (so 8 passes is out max probably id do would be 4 if used)

Whenever im in doubt of whether a movie will look good i just up the number of cds and hence compressibility (i try to keep to like 70% or more if that means 3 cds then 3 it is ) space is not an issue i just want a decent end product even if its 3cds instead of 2 or 2 instead of 1 etc.

also is there really much of a difference in using pshyco visual modelling (light , normal) in encodes.

I realise this is an Xvid forum but i thought id ask here as the questions are sort of related to whats in this thread

[begu]

Very nice comparison indeed. But to be honest, I didn't read it so carefully and I thought that no PP was used. I now recall that RV9 postprocessing can't be turned off, stupid me, bad memory.

The reason why I did think that PP was not used, because I watch my movies through Sigma's Xcard connected directly to TV. It doesn't use PP at all (in the way that PP is used usually by directshow decoders). So I use PP very rarely, I'm just not used to think about it. And because the output quality is very good (sharp, and good contrast comapred to VGA) all those little mosquito noises and macroblocks are very clearly seen, if they exist.

Becasue of the way I watch my movies, I need the encoding process to be as good as possible. So bad macroblocks should be avoided during encoding. To sum up it would be nice someday to have additional few pictures here and there to be presented with no PP. I know that the major part of the people watch the movies (encoded wit the 'divx'-style codec) using PP. And usually the movies are watched on the VGA or the TV output. And because of that there can be many resizing operations before the video gets to screen (decoder output (512x384) -> vga in tv mode (800x600) -> TV set (640 x 480 etc.))

Of course for example the matrox cards (I have g550) have good output and it supports directly TV res., but I think that many out there have old 'gf'-type cards and they doesn't give so good quality. And finally, in the future there will be more and more 'xcard' type 'livinroom' playback devices that support mpeg4 and similar. Generally they don't have so much and powerful postprocessing, but that maybe can be corrected in the future (more options for the user and firmware updates). And after 1-2 years we all can have dvd burner for decent price and can fit 2-3 movies with very high bitrate to one disc, and so making PP not so important. But after all at the present moment the comparison is very informative, thanks for Doom9 :)

[mf]

Originally posted by Doom9
And all the VCD/SVCD/DVD-R folks have a really strong filter called TV.
You know? Since I first used TV-out i was really disappointed with the quality I got. I see all the artifacts that I see on a PC monitor, and even more because of bad PC-color range to TV-color range conversion. Is my TV that good, or something?
Originally posted by Angrychair
Sticks and stones, my thin-skinned friend, sticks and stones. How about striking yourself for that 'bast***' comment?

What you like better isn't really relevant and in fact really throws the entire result into question. You're now letting your opinion skew the entire test.
There have been lots of undeserved strikes on this forum, but this one was justified imho. Questioning the accuracy of a test is fine, but taking words in your keyboard(:D) like foolish and stupid is just asking for trouble. Doom9 has clearly stated that his test is meant to take subjective quality (as he is the only reviewer that means what he likes better) in account, with the settings that average doom9-reader will have set for playback. Sorry that you aren't part of that average. (Just like I'm not part of that average either, btw, as I have extreme playback filtering settings on to make crappily encoded fansubs (90% of them) look watchable. I'm not asking him to set max postprocessing with a Conv3D at playback for his futurama test either.)

[dragongodz]

Doom9@ interesting read as usual.

to all the people complaining that they would have done it different well go right ahead. then post a link to the website for it. then wait for everyone else to pick it apart and point out how they would have done it differently. :devil:

its especially rich for people with single digit posts to be complaining aswell. try to contribute a bit yourself and maybe YOUR opinion will be taken seriously. :)

[Nic]

I agree with dragongodz there. And this thread is very offtopic now for this forum.

I thought Doom9's comparison was excellent. But like anything can be picked apart.

Im going to close it, Doom9 will re-open it if necessary, but for me this is too offtopic, and critising (even when constructively) is bad form, unless you have experience of creating the same kind of comparisons.

-Nic