Objective quality

[gronne]

Is there any way you can objectively say how good the quality of e.g. x264? And I don't mean by making a poll what people think, but if there's e.g. any software that can show how good the quality is compared to the source?

[Dark Shikari]

You can use PSNR, which is totally useless for judging the effectiveness of any psychovisual optimization.

You can also use SSIM, which is almost as totally useless as PSNR.

Given that both claim that this image (http://i27.tinypic.com/a4ldtj.png) is about 20% worse than this image (http://i29.tinypic.com/344fpd3.png) (PROTIP to the regulars here: this is a preview of x264's upcoming psy RD mode!), I can quite clearly state both metrics to be useless ;)

There isn't much better. It's hard for an algorithm to approximate the human visual system.

[gronne]

Ok, thanks for the fast answer! I was thinking about some of these sort of programs, but if you claim they don't really work, it's too bad. You, as a developer, how can you always know that you make progress all the time you change something? Do you have eagle-sight? :)

[gronne]

And I agree, the first image is not even nearly as blurry. The details in the first are far better.

[Dark Shikari]

Ok, thanks for the fast answer! I was thinking about some of these sort of programs, but if you claim they don't really work, it's too bad. You, as a developer, how can you always know that you make progress all the time you change something? Do you have eagle-sight? :)For non-psychovisual improvements like a better motion search or more RD functions, I use PSNR at a specific bitrate (both test clips encoded at the same target bitrate).

For psychovisual optimizations I ignore PSNR/SSIM and use only my eyes.

[Avenger007]

Given that both claim that this image (http://i27.tinypic.com/a4ldtj.png) is about 20% worse than this image (http://i29.tinypic.com/344fpd3.png) , I can quite clearly state both metrics to be useless ;)
Given that some of us still use SSIM as a guideline for quality, why would SSIM be about 20% less for the first image? How does the original image look?

[Dark Shikari]

Given that some of us still use SSIM as a guideline for quality, why would SSIM be about 20% less for the first image? How does the original image look?The reason is because SSIM is, in some sense, just a variance-weighted PSNR; as such it falls into many of the same traps that PSNR does.

[J_Darnley]

Original image (http://img144.imageshack.us/img144/4527/orig0le6.png).

The reason is because SSIM is, in some sense, just a variance-weighted PSNR; as such it falls into many of the same traps that PSNR does.
Uhh.. how does x264 change 300 into parkrun?

[Dark Shikari]

Uhh.. how does x264 change 300 into parkrun?Oh wow, I confused the two images I originally posted :p

I thought I had used the "300" samples here but then I realized I used parkrun... oops. But since I have that image uploaded, here's the three images:

Worse PSNR/SSIM-wise (http://img519.imageshack.us/my.php?image=psygb2.png)
Better PSNR/SSIM-wise (http://img204.imageshack.us/my.php?image=nopsytg8.png)
Original (http://img144.imageshack.us/img144/4527/orig0le6.png)