Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 21st September 2018, 16:16   #1  |  Link
Forteen88
Herr
 
Join Date: Apr 2009
Location: North Europe
Posts: 556
Which is a good image-quality metric?

Hi, I wonder, which metric is good for measuring image-quality?
I've seen some people often use "Avg QP" as image-quality metric (with low Avg QP as better quality), but isn't SSIM a better metric than that?

Yeah, I know the eyes are the best image-quality thing to use, if you got good eye-sight.

Thanks in advance.

Last edited by Forteen88; 21st September 2018 at 16:25.
Forteen88 is offline   Reply With Quote
Old 21st September 2018, 16:19   #2  |  Link
nevcairiel
Registered Developer
 
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 10,340
Avg QP is technically not a quality metric, but a compression metric. There is a relation to quality of course, but its not its primary purpose.

Metrics like SSIM are far better for that. Or Netflix' VMAF.
__________________
LAV Filters - open source ffmpeg based media splitter and decoders
nevcairiel is offline   Reply With Quote
Old 21st September 2018, 16:49   #3  |  Link
alex1399
Registered User
 
Join Date: Jun 2018
Posts: 56
For comparison the PSNR of luma component (Y) only is quite fast and not bad. The x-y plot of SSIM (Y) to PSNR (Y) doesn't have much outliers and they have some strong positive correlation. In addition, it's great on detecting minor image shift which eyes cannot differentiate.
alex1399 is offline   Reply With Quote
Old 23rd September 2018, 19:45   #4  |  Link
zub35
Registered User
 
Join Date: Oct 2016
Posts: 56
If your purpose is to assess the quality, with minimal changes from the original (visually lossless (equivalent jpeg qualities 90+))

butteraugli - https://github.com/google/butteraugli

Other metrics of quality measurement will lie, because of the application of psychovisual algorithms in modern codecs.

Last edited by zub35; 24th September 2018 at 00:38.
zub35 is offline   Reply With Quote
Old 26th September 2018, 00:47   #5  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by nevcairiel View Post
Avg QP is technically not a quality metric, but a compression metric. There is a relation to quality of course, but its not its primary purpose.

Metrics like SSIM are far better for that. Or Netflix' VMAF.
x265's Rate Factor is also more subjectively correlated than QP, since it is basically QP with an image complexity offset.

VMAF is the least-bad metric we have today, but it is still limited by being a per-frame metric, which then gets averaged across frames. So an encode with frames ranging from 70-80 and an encode with frames ranging from 50-100 would both average a VMAF of 75. But the one going from 50-100 is going to have a lot more poor looking frames, and would deliver a much poorer subjective experience.

VMAF is also calculated relative so source frame sizes, so the same VMAF score from a 720p source and a 1080p source will mean quite different things. The "right" way to do it would be to scale source and encodes up to 1080p to run the VMAF comparisons. But that really increases the calculation time for VMAF.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 29th September 2018, 08:58   #6  |  Link
Forteen88
Herr
 
Join Date: Apr 2009
Location: North Europe
Posts: 556
Quote:
Originally Posted by benwaggoner View Post
VMAF is the least-bad metric we have today
Why isn't VMAF implemented in x265?

EDIT: I thought more about licensing, if it's any problem implementing it into x265 because of that.

Last edited by Forteen88; 17th October 2018 at 22:48.
Forteen88 is offline   Reply With Quote
Old 2nd October 2018, 03:42   #7  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by Forteen88 View Post
Why isn't VMAF implemented in x265?


Because no one has checked in code to do it ?

VMAF elementary metric calculations should be MUCH faster to do in x265 than ffmpeg because the source and output frames are both right there in the CPU cache together.

Since the VMAF model itself changes, actually calculating VMAF itself wouldn’t be as valuable as calculating the elementary metrics, which would allow improving VMAF accuracy for existing encodes whenever the machine learning model is updated.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 17th October 2018, 22:47   #8  |  Link
Forteen88
Herr
 
Join Date: Apr 2009
Location: North Europe
Posts: 556
By the way, I read that VQM is also a good quality-metric,
Quote:
This is also another argument against using simple metrics such as PSNR or SSIM to judge video quality: they cannot take into account perceptual effects like motion, since they only look at individual frames. More perceptually-based metrics like VQM or VMAF are better choices for evaluating video degradations.
https://slhck.info/video/2017/02/24/crf-guide.html
Forteen88 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 08:57.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.