libavcodec mpeg4 codec disqualified

[Doom9]

I'm sorry to announce this but due to what has transpired in the mencoder user list, especially the completely unprofessional and destructive behavor of both libavcodec developer Michael Niedermayer and mplayer developer Rich Felker, libavcodec has been disqualified from participating in both this year's codec comparison and any future ones.

I have never seen behavior from any participant in the past that even comes close to this and quite frankly I cannot work any further with people who claim to hold absolutel truths and are completely unwilling to accept different opinions and differentiate between statement of fact from my own perspective and unwarranted attacks on their person.

If you care to read up on it, you can do so here (http://news.gmane.org/gmane.comp.video.mencoder.user) but that's it for me. I have better things to do than deal with zealots. I hoped to conduct this test in a quiet and professional manner and not resort to such drastic measures, but enough is enough. It is sad to see the comparison suffer, but it's just history repeating itself.. similar things happened the last time I included a libavcodec in my comparison and at some point I just have to cut my losses.

[Sagittaire]

I have never seen behavior from any participant in the past that even comes close to this and quite frankly I cannot work any further with people who claim to hold absolutel truths and are completely unwilling to accept different opinions and differentiate between statement of fact from my own perspective and unwarranted attacks on their person.

As I say in my PM their setting (libavcodec developer Michael Niedermayer and mplayer developer Rich Felker) are very strange ... lol
- low setting for vqcomp
- No adaptative bframe with bframe/ratio/offset = 2/1.25/1.25

With these "best setting" the doom9's conclusion are IMO the true. No adaptative bframe with low bitrate variability are certainely not good for high motion matrix scene ... :eek:

It's easy to see if RC is bad : make sample with high motion scene and if sample for libavcodec is always smaller than XviD or DivX6 then libavcodec's RC setting are bad for high motion scenes ...

[Doom9]

this isn't about the settings though (as far as I can tell that was a community effort and it's a shame to see it go to waste), the codec wasn't doing so bad with the settings as it is (you can read up the threads to see who suggested what and why a certain setting was adopted.. your vqcomp comment actually rings a bell.. I think there's quite a few people who think it did more harm than good). But the ASP category will remain interesting anyway with 4 participants, only two of whom will make it to the final round.

[Atamido]

especially the completely unprofessional and destructive behavor of both libavcodec developer Michael Niedermayer and mplayer developer Rich Felker
I don't know why Michael would be acting bad, but Dick Felker hasn't been taken seriously in years. He is a troll and should be automatically ignored from any conversation.

I think if you go back and re-read the postings, without looking at Dick's, then you'll see it's not so bad at all. While Michael can be a little unbending in his views, I would venture to say that most projects suffer from at least a few of these people, not the least if which are some Doom9 mods that I've seen in the past.

To exclude an entire category just because of the actions of one stern developer, and one troll, would be a greater disservice to the Doom9 community than it would be a statement against them.

[ak]

I'm sorry to announce this but due to what has transpired in the mencoder user list, especially the completely unprofessional and destructive behavor of both libavcodec developer Michael Niedermayer and mplayer developer Rich Felker, libavcodec has been disqualified from participating in both this year's codec comparison and any future ones.
Like forever?

Lol, so childish. I for one don't see anything completely unprofessional and destructive in libavcodec developer Michael Niedermayer comments. As for mplayer developer Rich Felker... http://article.gmane.org/gmane.comp.video.mencoder.user/2328
Ther's a certain prejudice from your side as well, at least to my liking.

Well it makes the whole thing much more windows-centric, hense, I dare to assume, much less interesting to non-windows crowd. Like if anyone cared.. :rolleyes:

[Doom9]

uh, xvid is OS agnostic, so is x264... But if you mean less interesting for those that hate codecs that do well on Windows, then by all means you're correct ;)

And by the qualification results, lavc doesn't stand a chance against XviD anyway.. no big loss really, and certainly not a disservice. If you want to be shrugged off and not taken seriously if you ever have a problem with lavc, you're better off using XviD anyway.. there are people that actually bother to verify before discarding input.

[Atamido]

And by the qualification results, lavc doesn't stand a chance against XviD anyway.. no big loss really, and certainly not a disservice.
So you say, but I thought the point of including the images was that much of the review is subjective to the reviewer. Am I mistaken? Wouldn't it be more relavant to include the information, state that you don't believe it lives up to the standard, and let other users independantly verify from viewing the images?

If not, then wouldn't it be much faster and less bandwidth intensive to not include results from any except the top codecs?

[Doom9]

the images are there, are they not? I wasn't aware removing anything.
But to be able to properly tell, looking at screenshot is the wrong approach anyway. Codecs are evaluated as described by Q20 in the FAQ, and you have the settings to make your own test encodes and see with your own eyes.

[ak]

uh, xvid is OS agnostic, so is x264... But if you mean less interesting for those that hate codecs that do well on Windows, then by all means you're correct ;)
Ok, I somehow missed x264, was thinking in ASP terms. Dunno, I for one don't feel for any codec nor hate, it's just that if you can't try a codec, it makes it a less interesting. Would you ever consider lavc if it wasn't cross-platform (read running on Windows)?

[hellfred]

Hi Doom9 and other forum users
I hope you sleep one night over the discussion running wild on the newsgroup and rethink if you really want to disqualify libavcodecs MPEG4-ASP.
Rich Felker is a hotblooded person and very convinced of lavc's superiority to any other MPEG4-ASP implementation, but on the other had he is always trying his best on mplayers user mailing list.
For Michael, i have to say I could not understand your claim of him being/having "unprofessional and destructive behavor" untill the very last mail he send at Date: 2005-12-19 16:26:00 GMT.
In my opinion, till this very last mail, all his comments on the thread were as passive/reserved as possible. Maybe the reason for my opinion is that both him an me are native German speakers.
I do not know what unpleasant or offending flamewars went on in the past years, or if there are other offending contibutions by michale in another thread than the one I read ( I read "new doom9 codec comparission (submission)" on the given link), and I base my comment only on these mesages.
Even on the last mai, he keeps on suggesting solutions, trying to retrace/correct statements of his own that might have been mistakeable or easily missunderstood or expressed unskillfull.
I would feel it a shame to disqualify the codec only because of the words exchanged between you an Michael an that newsgroup thread, as I do like to encode using mencoder, both for its speed and the wide range of input you can throw on it. So i was hoping for a good comparison of the benefits and drawbacks when using xvid or lavc for my encodings.
Even if insults from former comparisons are still hunting you, please cool down, read Michaels commets once more and please rethink your decision.

Stefan

[Doom9]

Would you ever consider lavc if it wasn't cross-platformOf course not because the comparison takes place on Windows. I thought that was kinda obvious. But there are a lot of codecs that have their roots on Linux, starting with XviD, x264 and dirac. And I get the feeling these days Theora development is also Linux based. All those codecs are included. It's mainly the commercial variety that is Windows centric.

[Sagittaire]

1) I think that libavocodec setting are simply not the best ... and it's not doom9's problem because doom9 consult developper for that ... lol

2) there are no decoding problem with libavcodec stream under windows (ffdshow, xvid or divx PP0 decoder done the same Overall PSNR than internal OPSNR mencoder calculation)

3) with the good setting libavcodec, xvid and divx6 done very close OPSNR but SSIM are very better for XviD and DivX6

[Manao]

especially the completely unprofessional and destructive behavor of both libavcodec developer Michael Niedermayer and mplayer developer Rich FelkerI cannot work any further with people who claim to hold absolutel truthscompletely unwilling to accept different opinions and differentiate between statement of fact from my own perspective and unwarranted attacks on their personMichael is none of that. I've followed quite closely this thread, and I wonder how you can say that Michael's behavior was unprofessional and destructive. He's been quite the contrary in fact. He's always backed his claims, most of which concerned lavc's decoding capabilities / issues. He's a bit twitchy when it comes to ffdshow's lavc implementation, but that's to be understood because he's got no way of control / feedback / testing on it.

Rich is another story, but disqualifying lavc because of him would be a shame, imho.
libavcodec has been disqualified from participating in both this year's codec comparison and any future ones.Aren't you overreacting ? I know if you're not used to Rich, you can find his comments quite displeasing, but he has nothing to do with lavc, and he's an easily spotten troll. So I stand by hellfred and ak.
but Dick Felker hasn't been taken seriously in years. He is a troll and should be automatically ignored from any conversation.Rah, it has been a long time since I didn't have that good a laugh so early in the morning. Thanks a lot for making my day :)

[IvS]

I don't understand any of this either.

Doom9: If you could explain to me and everyone why exactly Michael caused you to disqualify the libavcodec encoder, forever, that would be good, since currently I really don't understand how such a rationale was made.
If it turns out that Rich is the main person that caused it, I still wouldn't understand how a person who has nothing to do with the encoder's development caused it to get disqualified, forever.

[videomixer9]

Honestly, imo Doom9 seems to be the arrogant part here who wants to select his audience and doesn't wanna hear any criticizing or whatever. I don't see that much arrogance on the side of those two ppl mentioned, at least not more than ppl usually tend to have in the open source scene. IMO it's sad to see ppl disqualifying others work just because they don't like the way someone expresses himself. Ever since South Park I call this the Eric Cartman Syndrome ... it's the typical "screw you guys, i'm going home mentality" ... whoever does that first doesn't really seem to show his will on doing sth.

[Doom9]

Well. obviously none of you guys ever has made a comparison. Prior to this I have yet to experience that any participant, instead of accepting results of what is obviously a subjective comparison, starts attacking right away the minute they smell they're not going to come out on top. What kind of professional behavior is that? Did you see the XviD team do that when their codec wasn't so good yet? Did you see them do that last year when a better codec came along? Do you see the people from DivXNetworks complain that they have never won?
And what when you report a problem.. would you not expect that somebody look at your samples and screenshots and not fire back immediately with "it cannot be, you must be something wrong" without even looking that screenshots and samples I've taken extra for them just to make my point? This is the first time I've done the comparison 3 times over with 3 different decoders only to rule out any hint of an impropriety.

When I compare and make a ruling, I expect participants to respectfully accept that ruling, whether they agree with it or not. I'm not throwing dice, I'm doing the best job I can and wouldn't dare ever to claim X looks better than Y if I did not truly believe that. To attack that, to claim I'm wrong or doing things the wrong way, is unprofessional and I cannot work with such people. Big parts of what the comparison is about has not been published in such details but each participant got a memo.. it explains about playback in ffdshow (see how all my comments about directshow filters were casually ignored? I don't care about mplayer playback.. my memo clearly states that playback will be done via directshow and that the postprocessing filter will be turned on.. if you don't like it, you have the option of gracefully backing out by not making a submission in the first place, rather than make that submission, and then if you don't agree with the results, blame anything but yourself).

at least not more than ppl usually tend to have in the open source scene. This is why open source will never take over in certain areas.. the arrogance and religious zealousness will scare away the normal folks. Those people will need an attitude adjustment and I wonder how they can ever make it in the professional life.. I suspect they hold a position where their superiors are afraid to fire them because of the nasty things they could do (e.g. sysadmins).

You don't agree with that? Well open your eyes and look at the real world. Take soccer in Italy as an example. The clubs are hold at leats partially responsible for what their fans do. Racism and otherwise inappropriate behavior of fan groups has a direct influence on a club.. they can be fined, even banned from playing. It's no excuse, in professional or personal life to not keep groupies and team members in line. And it gets even worse if somebody is listed on a member page. If there are any Doom9 groupies (I doubt there's such a thing) that are out of line, let me know and I'll make sure to whip them into shape.

imo Doom9 seems to be the arrogant part hereYou have 12 hours to explain that or 30 days to enjoy your rule 4 strike. Make your pick.

I'll make your job a little easier:
Is it arrogant that if at the hint of any improprieties, you go back and retest to make sure your results are meaningful, and that in spite of people who cannot even spell the world tact?
Is it arrogant if when being told that X cannot be, you go back, take screenshots that prove this statement is false.
Is it arrogant that when confronted with yet another statement that what I'm saying (XviD decoded with the XviD filter looks better than decoded with ffdshow)is false, you go back again, cut samples, upload them and expect people to have a look at them BEFORE they make any futher statement?
Is it arrogant when you expect your participants that once submitted, they play by the rules they were given (e.g. the fact that postprocessing would be used was known, and from other participants I'm getting playback recommendations and recommendations for taking screenshot and how the filter reacts with certain settings instead of "this is unfair")?
Is it arrogant to expect that when given detailed information on how the test is conducted up front and given the opportunity to pick the settings to make a codec shine, to then be professional and courteous about the outcome and not put the blame on testing methods if you don't agree with the results? What's the point in giving people the opportunity to submit settings and builds to be tested if they cannot accept the outcome of what they submitted along with the information about the test they had upfront?

If so, then by all means I'm an arrogant prick.

P.S. It's interesting that in a private discussion, it turns out some of Michael's statements were indeed premature..

[IvS]

Doom9: All this still doesn't answer what Michael said that was so wrong which terribly offended you so much that you disqualified the encoder for this and any future tests.
Premature statements or not, the main thing which seems premature to me is this thread's creation with such an unexplained outburst.

[Lefungus]

I think the main error here was to use mplayer ML where no sensible discussion can be made due to one specific troll.
About the disqualification, well, what will it achieve ? People will say that you forgot the "superior" lavc and that your test is broken. It would be better to let's them argue about why xvid has comparable results, and *gasp*, even better.

[Doom9]

Any participant who goes on public record denoucning results without bothering to do proper research (especially when given sample material) is exhibiting extremely unprofessional behavior. I expect more from every particpiant. If confronted with facts, you don't denounce until you verify, and you verify using the same software, not on another OS and another player.

It is in no way different from me making any statement about any codec I have yet to test and any comparison I have yet to make.

This isn't about trolls, it's about the main codec developer denouncing methods he knew up-front (don't participate if you don't agree), and results (don't participate if you can't take not winning). Claiming it makes no difference with decoder is used when I retested 3 times, is just not appropriate unless you have bothered to reproduce this in the given scenario. I don't care if anybody doesn't have a Windows box available.. it's well known how these tests are being conducted. Rich can spew BS all he wants, unless the main codec developer joins in the unfounded argumentation and discarding without review line it's not really my problem, but the moment the person behind the codec starts with inappropriate behavior, that's when I have to draw the line. It's inappropriate to ask to use a decoder that makes other codecs look worse (in my eyes of course), it doesn't matter if that's not what your own experiments have shown. When confronted with those clips, the only thing to be done is go to a windows box and see for yourself, and only then make a statement. Or next thing I know, I have people from all the participants ganging up on me because their codec didn't win.. I just cannot allow this to happen and I will disqualify any other participant where lead developers ever exhibit the same behavior. That I never had to do that in the past just shows that there are a lot of people who have a little more professionalism.

Many here pick open source codecs over commercial ones not only for quality, but basically political reasons as well (commercial versus open source is such a decision imho). Likewise, you can opt to not use a codec because people working on it don't share your ideals.

[guada 2]

" I just cannot allow this to happen and I will disqualify any other participant where lead developers ever exhibit the same behavior ".

I think that everybody took note.

@ Doom9
Make what is good for humanity and not what you wish.
We are nothing without the others.
We have all sinned, never forgets it.


God is everywhere.

My regards DOOM9..

Mario.

[Inventive Software]

Wow! I just spent about 45 minutes studying the link Doom9 supplied at the thread opener, and I have to agree with him. I have a suggestion though: to keep everything fair, maybe the reference IDCT should have been used, as it is the original implementation. Please Doom9, don't take it the wrong way, it really is just a suggestion. :)

[Doom9]

to keep everything fair, maybe the reference IDCT should have been usedThat would raise the question which codecs allow a selection of idct in the first place.. And then there's probably the matter of idct mismatches between the mpeg2 encoder and decoder. I recall back in the flask day, THG raised the idct issue.. I did my own test and concluded that not using the reference mpeg-2 idct did not matter.. you could spot differences in screenshot but watching a moving video was another matter entirely.

[CREXbzh]

That would raise the question which codecs allow a selection of idct in the first place.. And then there's probably the matter of idct mismatches between the mpeg2 encoder and decoder. I recall back in the flask day, THG raised the idct issue.. I did my own test and concluded that not using the reference mpeg-2 idct did not matter.. you could spot differences in screenshot but watching a moving video was another matter entirely.

IDCT problem is not the main problem of ASP codecs, but it's one of those that are very annoying. I advise you read 15 reasons why MPEG4 sucks (http://guru.multimedia.cx/?p=10) which is a text written by Michael himself. In his own experience, ASP codecs can't be easily compared because of the lack of well defined things in the spec.

That pretty much means to me that ASP codecs are just to be considered as just a draft of "MPEG4 done right," aka MPEG4 AVC.

That means to me that, trying to find who is the best against all ASP codecs is not the kind of interesting question that I'd ask myself if I were to choose a video codec.
I say that off course now that I managed to get used to the AVC codec I use (x264) so that I know how to tune it to make my encode look beautifull.

Doom9, I'm really sorry to hear that things have not gone as well as they should have with the lavc crew. I hope that by the time you'd do another codec comparison, you'd have changed your mind, and that hopefully Snow will have made some progress.

Regards,

WJMP

[Sagittaire]

And before anyone comes argueing these are irrelevant because they don't
have the same size: that's the reason why I'm encoding the full movie and
looking at selected high motion and low motion scenes to ensure no codec can
get away focusing only on one type of scenes. So, as we can see, lavc gives
high motion scene less bitrates which I can spot quite clearly, whereas it
spends the saved bits someplace else that I cannot see. And I'm afraid rate
control is an important part of a codec.. as the VSS example clearly shows,
in a rather negative way.

As I always says ... RC setting are simply not good ... lol

- No adaptative bframe with 2/1.25/1.25
bframe with high quant in high motion scene ... :rolleyes:

- very low bitrate variabilty (vqcomp = 0.6)
it's good for AVC codec but certainely not for ASP. For example XviD and DivX use praticaly constant quant for PFrame in second pass (with ratio/offset for Iframe and Bframe and without vbv) aka equivalent vqcomp = 1.00 for lavc. IMO the best value for vqcomp is in [0.75;1.00] interval.

[Xayd]

IDCT problem is not the main problem of ASP codecs, but it's one of those that are very annoying. I advise you read 15 reasons why MPEG4 sucks (http://guru.multimedia.cx/?p=10) which is a text written by Michael himself. In his own experience, ASP codecs can't be easily compared because of the lack of well defined things in the spec.

That pretty much means to me that ASP codecs are just to be considered as just a draft of "MPEG4 done right," aka MPEG4 AVC.

That means to me that, trying to find who is the best against all ASP codecs is not the kind of interesting question that I'd ask myself if I were to choose a video codec.
I say that off course now that I managed to get used to the AVC codec I use (x264) so that I know how to tune it to make my encode look beautifull.

Doom9, I'm really sorry to hear that things have not gone as well as they should have with the lavc crew. I hope that by the time you'd do another codec comparison, you'd have changed your mind, and that hopefully Snow will have made some progress.

Regards,

WJMP

except there's no MPEG4 AVC hardware, and regular ole x86 PCs can't play it at DVD or higher resolutions/framerates (at least not those that are less than very cutting edge in speed, and therefore $$$).

therefore for those of us who are end users who are only concerned with what's usable on our current PCs and standalone devices, discussion of MPEG4 AVC at this point is purely academic.

[iive]

After reading the whole maillist, I can only quote Michael's words:

„the whole thing really is just a silly missunderstanding, due to everyone
saying one thing and meaning something slightly different“ ( link (http://article.gmane.org/gmane.comp.video.mencoder.user/2312 ))

Doom9, I think that you have overreacted and you have missed Michael's final reply (http://article.gmane.org/gmane.comp.video.mencoder.user/2330 ).
That's why I quote the most important part of it:

„iam NOT claiming that ANYTHING in your conclusion about the ENCODING
comparission is inaccurate, false or anything!“

For Michael your claims that one standard compliant decoder looks better than the other means only one thing: There is some bug. So he tries to find it out. And indeed, he had later found 2 bugs in xvid decoder <1> (http://article.gmane.org/gmane.comp.video.mencoder.user/2337) <2> (http://article.gmane.org/gmane.comp.video.mencoder.user/2345).

Doom9, I do believe you must read the thread again and try to understand everybody's point of view.
If you still find something offensive I would like you to quote it here, because your accusations are not very specific.

Here is a quick link to the mail thread with Doom9 (http://thread.gmane.org/gmane.comp.video.mencoder.user/2245)

[IvS]

iive: all i can say is don't expect anyone to admit anything ;) and praise to Michael for clarifying this.

[Doom9]

So he tries to find it out. And indeed, he had later found 2 bugs in xvid decoder <1> <2>. Yeah, after I disqualified his codec.. if that's the only thing that can get a guy to get off his high horse and don't just make claims but starts verifying them... I provided screenshots and samples to prove my point that decoding XviD via XviD or lavc looks different.. the only thing to do when that is done is to verify, not continue making claims that later turn out to be wrong.

[PatchWorKs]

...sincerly, can't understand this discussion.
Tests are under Windows ? So encode by ffdshow (which uses libavcodec).
Settings are undertandable (and usable to all).

Just my opinion, anyway. :helpful:

[hellfred]

@Doom9
Thanks for changing your mind, bugging Michael to acually verify your examples and keeping lamp in your comparison.
:thanks: :thanks: :thanks:

[IvS]

Doom9: I just don't get it. If there's any "high horse," Michael is not the one sitting on it. He obviously knows much more than you and most people about MPEG-4 video, but other than the various efforts to call him a snob without saying the word in this thread, I haven't seen anything else from you like maybe trying to help solve the problem or even accept the possibility there is a problem, considering he can't be someone who makes claims from thin air.
I'm not surprised that it took him a long while to find the cause for the strange issue, since he had no one to actually offer help with it, and it's not like very knowledgable video coding people grow on trees and can jump down from them to help him, yet you make the ridiculous claim that he "got off his high horse" and solved the problem, as if it took some seconds.
The fact remains that you disqualified the work of a talented individual who's contributed immensely to the "open source community" (or, people everywhere) due to lack of tolerance for contradicting ideas to yours, and in the end it turns out yours were wrong.

[Doom9]

The fact remains that you disqualified the workYou gotta love clueless people like you.. obviously you never read the main round article did you? Or hellfred's comment should also have lit a lightbulb somewhere..
and in the end it turns out yours were wrong.Quote the opposite.. I made a statement (lavc decodes my xvid clips differently than lavc: http://article.gmane.org/gmane.comp.video.mencoder.user/2245) that Michael discarded (http://article.gmane.org/gmane.comp.video.mencoder.user/2293 & http://article.gmane.org/gmane.comp.video.mencoder.user/2294 ), and asked to provide samples for files that are decoded differently, which I did: http://article.gmane.org/gmane.comp.video.mencoder.user/2304 and http://article.gmane.org/gmane.comp.video.mencoder.user/2305 and which were subsequently ignored, up until the point where I pulled the plug.
And when he finally bothered to look, he found out that indeed I was right and his statement is equal decoding was false (http://article.gmane.org/gmane.comp.video.mencoder.user/2337).

I haven't seen anything else from you like maybe trying to help solve the problemHow about providing samples and screenshots when being asked to? How would you feel if you're being snubbed after making an additional effort to help out? Not doing so tells me the following "You ain't got no clue, I don't need to bother because I'm right anyway". That's arrogant and inappropriate, especially towards somebody who happens to know what he's doing. It's not like this is the first time I'm encoding video after all, and I expect from every participant that they take my input seriously.. and that has never been a problem with any other participant.

And if you still don't get it.. just try to annoy me a little more <evil grin>

[Inventive Software]

Woah, Doom9, calm down! We all know you are the all powerful here, but may I make a suggestion?

Read, re-read and re-read again everything that has been posted about this situation, starting with your link to the mail thread. Then try to see it from the point of the devs. They're the ones that provide the software, they are the ones that are trying to solve the problem. It may be that Michael just took longer than you were expecting. (Don't quote me on that, as I may be entirely wrong!)

Did you try other implementations of lmp4? AFAIK, ffdshow and ffmpeg are pretty competent. I've used ffdshow without many problems, though I'll admit I'm in PAL land, and that may explain a few things.

Now after reading it again, I'm slightly confused. This seems to have been blown way out of proportion, partly by you (Doom9) and also by the people who want to stick up for mencoder's lmp4. Think carefully about what you wrote, and whether it could have been interpreted differently by those who "don't read properly".

[IvS]

Doom9: Of course you noticed there is a difference, but you never agreed it could be because of a bug. You in fact expected him to accept bugginess without even knowing it or trying to verify it.
"Michael discarded"? Well, from the link you posted:
"files with b frames have different md5 sums ill try to figure out why"
"if anyone has a non broken file which decodes binary differently with lavc & xvid (assuming xvid idct, bitexact mode, w&h %16 == 0 and such) then iam interrested in seeing it"
Subsequently ignored? I don't see anyone ignoring your samples. He admitted he forgot to test a sample. Sue him. Also, he's the only one who actually tested and found the XviD bug, not anyone else. You expect Michael to do pretty much all the testing work and then blame him for not detecting a bug with software that isn't even his on time.

"And if you still don't get it.. just try to annoy me a little more <evil grin>"
Since this is obviously a threat to ban me, god knows why, I guess because my ideas are too snobbish for you, by all means ban me, that would be a very sensible thing to do.

[Doom9]

by those who "don't read properly".Ignorance is no excuse, especially in light of rule 1.

Sue him. No, it's called disqualification ;) And since it has apparently been fixed, I guess it's time to encode again.. too bad I already got rid of the samples. I cannot imagine this can explain the difference though, especially in the light that there is no XviD deblocker in ffdshow, there's only the standard mplayer one.

And there's that other matter for making claims that the ateme decoder unfairly puts lavc and only lavc at a disadvantage, or a larger one than any other codec, without any proof whatsoever. It's a whole pattern of sore loser behavior. Everybody is out to get us.. buhu, even though we know we're the best. If I were to cater to that attitude I can close shop right now.. there are always #contestants - 1 losers in every comparison.

[iive]

OK, I will try to clarify the misunderstanding. Here is my point of view.

This is (http://article.gmane.org/gmane.comp.video.mencoder.user/2291) your first post, where you say that xvid and lavc decoded images look differently.

In this (http://article.gmane.org/gmane.comp.video.mencoder.user/2293) post Michael runs an experiment.

mplayer ~/mpc2/samsung_t20/home/michael/videos/310058.avi -lavdopts idct=14 -vo md5sum
mplayer ~/mpc2/samsung_t20/home/michael/videos/310058.avi -vc xvid -vo md5sum

These commands run lavc using XviD IDCT routine (yes it is ported from XviD) and xvid itself. The output frames are fed into md5sum (http://en.wikipedia.org/wiki/Md5sum) checker. Then by comparing the checksums one can determine if the frames are binary identical.
Of course the problem here is that Michael doesn't have the encodings you have. So instead he tries one made by himself. All the streams he produces decode binary identically on both decoders.

To prove your claims you send (http://article.gmane.org/gmane.comp.video.mencoder.user/2304) him pictures of both outputs. Unfortunately you had turned postprocessing on.

Postprocessing is not essential part of video decoding, but optional. Postprocessing uses internal data to remove some of the defects caused by quantization and transformation and as the name suggests postprocessing operates on the already fully decoded images. The need of internal data (MB quants) is the reason why it is usually included in the codec binary, but e.g. MPlayer allows export of this data to the filters and using of different custom postprocessing filters. The postprocessing in H.264 that is essential part of the decoder is called loopfilter.

In short postprocessing modifies the image in order to improve it. This means that Michael cannot determine if the differences in both images is caused by different postprocessing algorithms of by difference(bug) in the decoding process. This is what Rich tries to explain here (http://article.gmane.org/gmane.comp.video.mencoder.user/2306).
Don't forget that ISO-14496-2 defines the decoder and the only thing that could differ is the IDCT precision, but we already know how to make both decoders use same (XviD) IDCT.

Here (http://article.gmane.org/gmane.comp.video.mencoder.user/2305) you do the right thing, provide the encoded samples. Unfortunately the file seem (http://article.gmane.org/gmane.comp.video.mencoder.user/2307) to be damaged and unplayable.

you provide another sample file, but it seems that (probably) Corey has given you wrong options („mencoder -of rawvideo“ – is used for storing uncompressed raw rgb/yuv images) but you seem to be able to play it with mplayer. Anyway few hours later without anybody answering to that mail, you send your [url=http://article.gmane.org/gmane.comp.video.mencoder.user/2326]final one (]Here[/url).

Later Michael finds out your example file. Now he can run the md5sum check again and find frames that decode differently. Then he can output their images. By comparing the output images he can find the different area and then locate the MacroBlock(s) that decode differently. Then he can inspect the properties of that block and inspect the source code to determine why it behaves differently.

What I see here is Michael trying to extract the necessary information from you to nail the bug.
You take this to be an insult of your codec comparison methodology.
This really is just a silly misunderstanding.

[Doom9]

Unfortunately the file seem to be damaged and unplayable.That's not quite true.. only the #§°@#°@# of a player called mplayer cannot handle it. Others do just fine, including the very Linux centric VLC. And needless to say the problem started in the mencoder backyard.. it cannot properly mux raw streams into an AVI.. mplayer may be able to play those but they are seriously bugged.

Corey has given you wrong optionsNo he did not. Since I was using MP4 there's no reason to output to AVI, raw will do just fine.

I'm no expert in mplayer, but if those two commandlines you posted are the ones to "verify" if the output looks the same, then that's completely and utterly useless. The fact that postprocessing (deblocking to be precise) was going to be used was known well in advance and communicated properly. By my understanding of the manpage, in order to use deblocking, the following options would have to be added: " -xvidopts deblock-chroma:deblock-luma" and for using libavcodec, the option would be "pp " and then something thereafter.

I'm currently ripping the DVD again so that I can make another comparison with XviD 1.2 (I take that over the 1.1 final to make sure the fix is included).

[Manao]

And there's that other matter for making claims that the ateme decoder unfairly puts lavc and only lavc at a disadvantage, or a larger one than any other codec, without any proof whatsoever. It's a whole pattern of sore loser behavior. Everybody is out to get us.. buhu, even though we know we're the best. If I were to cater to that attitude I can close shop right now.. there are always #contestants - 1 losers in every comparison.That was definitely Rich, not Michael. Only once did Michael say 'ateme might be

[iive]

And there's that other matter for making claims that the ateme decoder unfairly puts lavc and only lavc at a disadvantage, or a larger one than any other codec, without any proof whatsoever.
This is easy to prove. Imagine that Ateme decoder also has 2 bugs that affect other codecs. How are lavc, x264 and xvid people supposed to find out why the output doesn't please you? Ateme is closed source and there is no way to check its source code.

[Manao]

And there's that other matter for making claims that the ateme decoder unfairly puts lavc and only lavc at a disadvantage, or a larger one than any other codec, without any proof whatsoever. It's a whole pattern of sore loser behavior. Everybody is out to get us.. buhu, even though we know we're the best. If I were to cater to that attitude I can close shop right now.. there are always #contestants - 1 losers in every comparison.That was definitely Rich, not Michael. Only once did Michael say 'ateme might be a disadvantage', and that was after saying ' the main problem with lavc was and is the ratecontrol'.

Anyway, all the decoder stuff was quite beside the point. You stated XviD decoded itself differently than libavcodec, so it wasn't an absolute truth that using lavc is better to decode mpeg4. Yet, in the end, you used ateme's decoder, which - if not buggy - would give the same results as lavc. In the end, all come back to ffdshow being at the moment buggy when it comes to seeking accuracy - which definitely isn't the fault of ffmpeg guys.

[Doom9]

This is easy to prove.Actually you need the decoder for any proof. Alleging isn't enough and after all I did what can be expected in this case, I changed decoding filters to make sure there are no improprieties... that's when I first discovered (or rather, confirmed my lingering suspicions that I had prior to the comparison) that lavc decodes xvid differently. To date, all those idct mismatch samples are still indistinguishable when watching in full motion (or appear to, even Rich admitted that), except those that exhibit clear discoloration.. and that's the thing you spot right away. But if you have a clip that proves otherwise, please upload to rapidshare and share the link.
Only once did Michael say 'ateme might beCase and point.. you don't allege without proof, and you most certainly don't allege in a public place to begin with.. if he had any doubts.. he has my email.

Spinning evil conspiracy theories, x264 could've never won the codec comparison because evil ateme purposely cripples other content it decodes... right? Come on. And there's the matter of Michael asking for lavc to be used for all codecs.. knowing or not knowing it had or may still have problem with other content, while other contestants also asked for their own decoders to be used, that request never went along with "use ours for everyting".

Yet, in the end, you used ateme's decoder, which - if not buggy - would give the same results as lavc.I doubt that they use the same code, so it would be different... different from what XviD does as well. But different doesn't mean better or worse. To my eyes, prior to the XviD bugfix, with ffdshow with mplayer postprocessing versus XviD with its own postprocessing, the latter just looks better to me, and therefore using lavc to decode XviD would put XviD at a disadvantage, or give lavc an advantage. So much about fairness.. My first pass encoding with XviD 1.1 is already done.. I checked the CSV and it appears the decoder fixes have been merged into the 1.1 branch prior to its final release.

[Manao]

I doubt that they use the same code, so it would be different... If none are buggy, they'll give the same results ( dct mismatches apart ).And there's the matter of Michael asking for lavc to be used for all codecs.. knowing or not knowing it had or may still have problem with other content, while other contestants also asked for their own decoders to be used, that request never went along with "use ours for everyting".Well, there're pros and cons in his proposition.

Pros :
lavc is still the fastest mpeg4 decoder
lavc is definitely the most thoroughly tested ( vlc, ffdshow, ffmpeg, mplayer, xine, which create a huge base of users, most of whom are accustomed to report bugs when they notice them ), so the less error prone, and the most able to test the standard compliance of the codecs.
lavc is the most friendly towards other encoders, since it tries to match their iDCT ( note : if Ateme, DivX, 3ivX ... do that to, I've never heard them state it )
a single decoder -> less trouble


Cons :
FFDShow current seeking implementation ( or was it only for AVC ? )
Prevent companies / codecs from using their own postprocessing
Rich ( ;) )


So, ok, his claims has drawbacks. But testing standard compliance is important too ( as the XviD bugs showed ).

[iive]

I'm no expert in mplayer, but if those two commandlines you posted are the ones to "verify" if the output looks the same, then that's completely and utterly useless. The fact that postprocessing (deblocking to be precise) was going to be used was known well in advance and communicated properly.

Now I see the problem.

These checks have nothing to do with your codec comparison.

These checks could be used only for finding bugs.
These checks proved that they are not completely and utterly useless, because they were used to find the 2 decoding bugs in xvid. That was Michael's intention from the beginning.

[Doom9]

If none are buggy, they'll give the same resultspure decoding perhaps, but postprocessing is another thing entirely, and it's in the FAQ.. postprocessing is part of a decoder in my book (and with AVC, VC-1 and proprietary ones like RV10 this is most definitely the case). It's not part (or optional) as far as ASP is concerned, but since ffdshow turns it on by default.. as does the DivX filter (and ND I think).. it's pretty obvious that it's the most common configuration.

lavc is definitely the most thoroughly testedAre you sure about that? vlc, ffmpeg, mplayer, xine.. those are solutions for a very limited number of users. Even with ffdshow, that might be more popular but can it rival the installed base of something like the DivX codec or NeroDigital? I do have my doubts about that.. the whole MPEG-4 area is still often seen as a synonym for DivX, even though in this place we know this isn't true.. but in real life even I just use the term DivX.. my friends and colleagues know what that is.. they don't know and don't care about the differences between DivX, XviD, etc.

As far as speed goes, I can counter that with the non SMP optimization.. if you have a demanding clip, a slightly slower but SMP capable system could be able to play a clip without a glitch whereas ffdshow would choke once it makes out a single core ;)

These checks have nothing to do with your codec comparison.That's right.. I thought we were talking about that. Sure it's nice (well, not nice but productive) that bugs were found along the way, but we'll know the relevance with regards to the comparison once I have a look at the now encoded movie.

[Manao]

As far as speed goes, I can counter that with the non SMP optimization.. And I have to keep you awake till your encoding finishes ;) : http://forum.doom9.org/showthread.php?p=471101#post471101. Even a very good smp decoder won't scale well enough to fill that gap.
vlc, ffmpeg, mplayer, xine.. those are solutions for a very limited number of usersLess and less for VLC, which in addition is more and more becoming a viable player in the industry. And I added 'users more prone to make a bug report', which imho have to be taken into account as much as the user base : more 'knowledgeable' and 'open-source-aware' users.

For the postprocessing, however, I agree.

[skal]

Hi,
That was Michael's intention from the beginning.

at this point of the conversation, i'd really like to praise Michael's pugnacity when it comes to fixing bug! Thanks!
-Skal

[PatchWorKs]

Well, the core problem is platforms, i think (as always).

ffdshow is the windows implementation, lavc is... a library (more *nix oriented, AFAIK)

So IMSO (in my stupid opinion) just throw away lavc and keep ffdshow.

If lavc could gain better results in your tests, but can't due to ffdshow well...
it's not your problem !

Hope this helps to calm down the discussion. :thanks:

[temporance]

This is easy to prove. Imagine that Ateme decoder also has 2 bugs that affect other codecs. How are lavc, x264 and xvid people supposed to find out why the output doesn't please you? Ateme is closed source and there is no way to check its source code.
Seconded. This was my concern when I first read the test methodology. Ateme could, if they were so inclined, have included something in their decoder like:


if (AtemeEncodedBitstreamDectected)
{
doGoodPostprocessing();
}
else
{
makeSomeArtifacts();
doCrapPostprocessing();
}

Of course Ateme would never do this, but as iive points out, their decoder could have bugs with a similar effect.

Additionally this was a codec test, not an encoder test. I'm not sure if ASP codecs are optimized this way, but it's conceivable that the various encoders are tuned for best results with their own decoder's postprocessing.

So I'm glad ASP decoding was done using the codec-under-test.

[Manao]

I'm not sure if ASP codecs are optimized this way, but it's conceivable that the various encoders are tuned for best results with their own decoder's postprocessingThough ideally it would be welcome, afaik, no open source ASP codec takes its postprocessing into account. It even goes further with x264, where postprocessing is transformed into inloop filtering, and where it still isn't taken into account when making encoding decisions. As Doom9 test showed, you can however see that it doesn't hurt it.

[OvejaNegra]

I really don´t know why this happens. I you don´t like doom9´s codecs testing, well, make your own testing. In the end, you will use your favorite codec, for one reason or another. If you don´t like the conditions of the testing, make the yours with your own settings. I always make my own tests with my source, with my settings and take my own decisions. I´m the one who is encoding my own things!!

[lu_zero]

The point is that since the whole issue resulted in a disqualification+name calling from one side, the other side is trying to not be on the same level and is just explaining the misunderstanding.

Isn't ffmpeg developers' fault if another software had bugs, be it ffdshow or xvid. They tried to find out what was the issue and your screenshots weren't of much help.