Study comparing HEVC HM, HHI HEVC, x265, H.264 JM and AOM AV1

[x265_Project]

A very interesting study...
http://iphome.hhi.de/marpe/download/Preprint-Performance-Comparison-AV1-HEVC-AVC-PCS2016.pdf

[benwaggoner]

A very interesting study...
http://iphome.hhi.de/marpe/download/Preprint-Performance-Comparison-AV1-HEVC-AVC-PCS2016.pdf

Ah, nothing like the good old mean of PSNR values as our presumed high-accuracy proxy for perceptual quality! Plus turning off all psychovisual optimization, because they don't matter in the real world :angry:

It is surprising than AV1 does so badly on this, however, as the VPx series development was highly focused on PSNR for years.

[Blue_MiSfit]

Yeah... yawn.

I get the academic value of this comparison but it's so far divorced from the real world (VP9 losing to x264... wut) that I can't place much value in it, personally.

x265 is bloody impressive - no doubt :)

[easyfab]

A very interesting study...
http://iphome.hhi.de/marpe/download/Preprint-Performance-Comparison-AV1-HEVC-AVC-PCS2016.pdf

Not exactly the same conclusion as what you can see here :
https://bitmovin.com/bitmovin-supports-av1-encoding-vod-live-joins-alliance-open-media/

But i'm ok to say that at this moment x265 is really really good if you take the speed/quality ratio.

AV1 speed is too slow for the moment ( no optimizations )
And VP9 with the new MT option is now better than in the past, cpu usage from ~40% to 80% and x2 speed~for me , but quality/speed is not as good as x265 yet.

[CSMR]

Not exactly the same conclusion as what you can see here :
https://bitmovin.com/bitmovin-supports-av1-encoding-vod-live-joins-alliance-open-media/

But i'm ok to say that at this moment x265 is really really good if you take the speed/quality ratio.
In this study HEVC codecs do well compared with AV1. But among HEVC codecs, the HHI solution is way ahead: "HHI HEVC encoder provides coding gain of 12.7%, while the encoding is more than 10 times faster compared to x265". Another study included the Intel Media SDK and that was way ahead.

Overall HEVC does well against other formats for free use (e.g. with the x265 codec), but blows everything away for commercial use (with HHI or Intel Media SDK).

Of course it will be great if x265 can significantly narrow the gap with the commercial encoders, or alternatively if the commercial encoders release free versions for non-commercial use.

[benwaggoner]

In this study HEVC codecs do well compared with AV1. But among HEVC codecs, the HHI solution is way ahead: "HHI HEVC encoder provides coding gain of 12.7%, while the encoding is more than 10 times faster compared to x265". Another study included the Intel Media SDK and that was way ahead.

Overall HEVC does well against other formats for free use (e.g. with the x265 codec), but blows everything away for commercial use (with HHI or Intel Media SDK).

Of course it will be great if x265 can significantly narrow the gap with the commercial encoders, or alternatively if the commercial encoders release free versions for non-commercial use.
I am pretty cautious when trying to extrapolate from this kind of study. The test was only done in a particular config that isn't close to a real-world tuning. Fixed QP, no rate control! It seems like every block of every frame is identical, so no psychovisual tuning, no adaptive quant, no cu-tree, 1 sec fixed GOP, etcetera. So not particularly close to a real-world scenario. And there wasn't any subjective testing done.

Plus if you wanted to do a real placebo full test of x265, you'd include --tskip --cu-lossless --bframes 16 --subme 7. And (if --tune psnr doesn't turn them off) --psy-rd 0 --psy-rdoq 0 --aq-mode 0.

And comparing encoder speed in this whacked-out scenario is pretty meaningless. Placebo is slow by design, and not intended for any reasonable price/performance tradeoff. And turning off all threading makes it even less real-world relevant.

Moreover, PSNR in itself is the objective metric with the least correlation with subjective quality.

This is really more of an Excel comparison of a highly artificial scenario than actionable info. I'm not going to care much about a suggested video quality difference without seeing some video in a relevant scenario.

But such is the way of academic codec research. They do it this way because they've always done it this way.