Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
|
|
Thread Tools | Search this Thread | Display Modes |
28th April 2009, 14:38 | #1 | Link | |||
x264 developer
Join Date: Sep 2005
Posts: 8,666
|
The 2009 MSU H.264 encoder comparison: a complete crock (or not?)
It seems that MSU is interested in fixing this one; see this post for the latest information. Good on them for caring about the accuracy of their results. Note I made some mistakes in writing this post as well, partially due to the way in which the report was designed--see the previous link.
Quote:
__________________
Follow x264 development progress | akupenguin quotes | x264 git status ffmpeg and x264-related consulting/coding contracts | Doom10 Last edited by Dark Shikari; 5th May 2009 at 10:18. |
|||
28th April 2009, 14:53 | #3 | Link |
Registered User
Join Date: Nov 2003
Posts: 1,281
|
There seems to be very little ssim difference between the High quality and High speed presets.
edit: Could you post the command lines? And personally, I probably would have given them more than 24hrs to respond. But your right, left hand isn't talking to right hand.
__________________
http://www.7-zip.org/ Last edited by Audionut; 28th April 2009 at 14:56. |
28th April 2009, 15:08 | #4 | Link | ||
x264 developer
Join Date: Sep 2005
Posts: 8,666
|
Quote:
Here's the settings I have from my own notes (might be slightly off as we tweaked them a bit): Quote:
__________________
Follow x264 development progress | akupenguin quotes | x264 git status ffmpeg and x264-related consulting/coding contracts | Doom10 Last edited by Dark Shikari; 28th April 2009 at 15:11. |
||
28th April 2009, 15:27 | #8 | Link | |
x264 developer
Join Date: Sep 2005
Posts: 8,666
|
Quote:
|
|
28th April 2009, 15:59 | #9 | Link | |
Registered User
Join Date: Nov 2003
Posts: 1,281
|
Quote:
edit: SD 12fps. need I say more. edit 2: A target ssim + bitrate showing resulting speed of each encoder would make sense to me.
__________________
http://www.7-zip.org/ Last edited by Audionut; 28th April 2009 at 16:05. |
|
28th April 2009, 16:46 | #10 | Link | |
x264 developer
Join Date: Sep 2004
Posts: 2,392
|
Quote:
Target speed+bitrate and measure ssim (as was supposed to be done), is the more flexible. Target speed+ssim and measure bitrate is theoretically possible, but most codecs don't have a target ssim ratecontrol mode, so it would take many retries. Most informative would be a target bitrate and measure lots of presets to graph a whole speed-vs-quality curve. Last edited by akupenguin; 29th April 2009 at 03:56. |
|
28th April 2009, 18:17 | #11 | Link |
Moderator
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,738
|
Do you have access to the sources used? It would be interesting to rerun the test with PSNR-optimized settings.
That said, I don't think either PSNR or SSIM is that well suited to the LCD era, as artifacts in low luma are much more visible than in high luma. Perceptually uniform gamma is a lovely idea, but it died as a realistic assumption with the CRT. I'm glad that plasma appears to be on the way out at least; it had different but also bad low luma issues, and it stank to have to sweat both. What was their platform for perf testing this time around? A couple of years ago it was some ancient single-core box that simply wasn't a realistic optimization target anymore. |
28th April 2009, 18:35 | #12 | Link | |
Software Developer
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
|
Quote:
Since "--me tesa" is included in official x264, wouldn't it be logical to include "--subme 10/11" too?
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊ Last edited by LoRd_MuldeR; 28th April 2009 at 19:19. |
|
28th April 2009, 18:57 | #14 | Link |
Testeur de codecs
Join Date: May 2003
Location: France
Posts: 2,484
|
@ benwaggoner
SSIM can use luma masking and other mask if you want ... @ Dark Shikari Well unfortunaly use particular setting to optimized each test for x264 is perhaps not good for egality if you don't make that with the other codec too (I have serious doubt about that). I think that by definition developper (not only you) are not objective to evaluate test procedure and result. Like always best solution IMO is tradeoff: soft AQ and soft psy (but real visual gain) with really good PSNR (less than 0.5 dB delta with best possible result). Dark Shikari know certainely x264 profil for produce that.
__________________
Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9 |
28th April 2009, 19:33 | #16 | Link |
Registered User
Join Date: Apr 2004
Posts: 1,315
|
I know I criticize a lot but speaking seriously:
MSU's comparisons had very serious mistakes of concept in past. And they did again. For example they compared Divx with postprocessing filter enabled to Xvid withouth PP. It sounded like a joke but it's hapened. A lot of graphs, explanations and other stuff are all useless when a rude mistake is done. And, of course, people know about unreacheble (by any other commercial encoder) quality of x264 . Even more, ssim and psnr have nothing to do with outstanding x264's psy optimiziations. |
28th April 2009, 20:47 | #17 | Link | ||
x264 developer
Join Date: Sep 2005
Posts: 8,666
|
Quote:
Quote:
Also, "soft psy" doesn't exist in x264--psy-rd always decreases PSNR and SSIM at any strength.
__________________
Follow x264 development progress | akupenguin quotes | x264 git status ffmpeg and x264-related consulting/coding contracts | Doom10 Last edited by Dark Shikari; 28th April 2009 at 20:50. |
||
28th April 2009, 21:37 | #19 | Link |
Registered User
Join Date: Apr 2002
Location: Germany
Posts: 4,926
|
Indeed that seems to have been a perfect example of a internal communication failure (one hand doesn't know what the other want's), but not sure if it's that bad most know the problem between PSNR & SSIM relation so one that sees the results and looks @ them carefully is gonna realize the difference that x264 is actually leading @ the SSIM results while being behind @ PSNR
I guess we can mirror the Mainconcept Results 1:1 onto the last DivX Encoder, or did they used their AQ and Grain processing ?
__________________
all my compares are riddles so please try to decipher them yourselves :) It is about Time Join the Revolution NOW before it is to Late ! http://forum.doom9.org/showthread.php?t=168004 |
28th April 2009, 22:27 | #20 | Link |
Testeur de codecs
Join Date: May 2003
Location: France
Posts: 2,484
|
well in my test really soft psy setting --psy-rd 0.50:0.0 --aq-strength 0.50 --aq-mode 1 produce similar result than Mainconcept SDK with best PSNR setting. Perhaps really small visual gain (I don't make comparison) but you have really good PSNR with AQ and psy.
__________________
Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9 |
Tags |
x264 test |
Thread Tools | Search this Thread |
Display Modes | |
|
|