View Single Post
Old 30th October 2015, 19:07   #8  |  Link
vivan
/人 ◕ ‿‿ ◕ 人\
 
Join Date: May 2011
Location: Russia
Posts: 643
Compression ratio is basically a bitrate, and you can choose any bitrate so it's doesn't mean anything. However "bitrate required to get a certain quality" is one of the goals of codecs comparisons which is very easy to advertise and very hard to get correctly - many, many people have failed at that.

QP is an internal parameter, it gives some approximation of quality for a given codec.
QP of a single macroblock is basically how much information is lost. Average QP is a simple average for all macroblocks. When using CQP (Constant QP) all macroblocks are encoded with same QP (P/B frames have some constant bias, but that's not important now). But encoding them that way is not very efficient - some areas of image could be compressed more, while others should not. This is what CRF was created for and why using CQP is not recommended.

PSNR is objective metric that was designed to compare videos. It's the simplest and fastest, but it doesn't correlate with HVS (Human Visual System, basically what defines what we consider "good") well. SSIM is an example of a better metric. Still it's not perfect. In fact x264 has --tune PSNR/SSIM option that increases given metric by sacrificing visual quality.

So we are back to the most basic question: "How to perform video encoders comparison?"
Lossy video encode is defined by these 3 parameters:
- quality
- size (bitrate)
- speed
(for lossless there's no quality, this is why it's so easy with lossless - lowest size wins, the end)
Lets assume that we don't care about speed much, and will just use best possible settings that give reasonable encoding speed. Now you have two parameters: bitrate and quality. Comparing videos with different bitrate and quality doesn't make much sense (video with higher bitrate may have better quality, and it will not tell you anything), so you have to fix one. Fixing bitrate is easy, fixing quality is hard.
Okay, you have two encodes with the same bitrate, now how do you compare quality? And this is the hardest question, because it does not have the best answer.
One option is visual comparison - you compare different frames and you decide which version is better (simplest way to do this is to use avsp and avisynth, but you can do it in any way you want). The problem is that people have different opinions, and also different encoders (and even settings) produce different kind of artifacts (e.g. x264 blocks while x265 blurs).
Another option is using objective metrics. It's very easy to do, no subjective opinion is needed, you can make cool plots with percents, etc. Downside is that there's no perfect metric.

Results also will be different for different bitrate/quailty levels (at high bitrate one encoder may be better, while at low - another), and of course for different videos. So it's important to perform several tests.

To sum up:
Comparisons are hard, the easiest and fastest thing I could recommend is encoding at fixed bitrate (2-pass) with SSIM tuning and comparing SSIM score.
For "bitrate required to get a certain quality" test - do lot of encodes with different bitrates (for both encoders), pick ones with closest SSIM and compare bitrates.
Also check this report to see cool plots and meaning behind them.

Last edited by vivan; 30th October 2015 at 19:18.
vivan is offline   Reply With Quote