View Single Post
Old 5th May 2009, 10:03   #45  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,666
And I finally got a response one week later, so I'm not leaking anything

Turns out that they seem to be interested in not making this a crock, and there were some misunderstandings!

1. Their results were based on SSIM, not PSNR--it was just very not clear that this was the case (they're going to clear it up in the final report). The problem was rather that they screwed up in making their "conclusion" graphs, resulting in x264 losing in places it shouldn't have.

2. I completely missed one series of tests: the series of CIF "videoconferencing" tests. Mainconcept won many of these, especially at low bitrates. My suspicion is that because it was 1-pass encoded and the clips were extremely short, x264 chose too high a QP for the first I-frame, which was sufficient to butcher quality at ultra-low bitrates in nearly-static clips like Akiyo. Unfortunately these clips don't actually represent reality, since in reality, you generally encode more than 300 frames at a time with a whole lot more than one I-frame. Because their scoring system counts large wins so highly, a couple huge Mainconcept wins at low bitrates here was sufficient to budge the overall score a surprising amount in its favor.

Overall, it seems to be a case of insufficient information in the report and incompetence rather than malice. Let's hope the final report is better.

Edit: suspicions confirmed about Akiyo:

Quote:
17:24 < Alex_W> 1 pass:
17:24 < Alex_W> x264 [info]: SSIM Mean Y:0.9676780
17:24 < Alex_W> x264 [info]: PSNR Mean Y:39.636 U:42.036 V:43.483
Avg:40.416 Global:39.396 kb/s: 62.23
17:24 < Alex_W> 2 pass:
17:24 < Alex_W> x264 [info]: SSIM Mean Y:0.9790610
17:24 < Alex_W> x264 [info]: PSNR Mean Y:41.145 U:43.939 V:44.836
Avg:41.976 Global:41.953 kb/s: 62.19
17:25 < Alex_W> in the 1 pass encode the I-frame has an average QP of 41.87 while in the 2 pass encode it has an average QP of 21.63

Last edited by Dark Shikari; 5th May 2009 at 10:27.
Dark Shikari is offline   Reply With Quote