Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
1st December 2006, 19:39 | #1 | Link |
Registered User
Join Date: Apr 2004
Posts: 1,315
|
New metrics. Better than SSIM?
There is some information about new metric that potentionally as good as SSIM or even better. From NTIA.
http://www.via.ecp.fr/via/ml/x264-de.../msg00107.html http://www.its.bldrdoc.gov/vqeg/proj...nal_report.pdf http://www.its.bldrdoc.gov/vqeg/proj...nal_Report.pdf Had anybody any previous experience with it? |
2nd December 2006, 15:54 | #2 | Link |
Registered User
Join Date: Mar 2002
Posts: 1,075
|
Nope, but I don't see other error metrics being implemented in a hurry. SSIM is popular not simply because how effective it is, but also because of how elegant it is.
The metrics which are better than SSIM tend to be highly complex and a pain to implement. |
3rd December 2006, 01:46 | #3 | Link | |
Registered User
Join Date: Apr 2004
Posts: 1,315
|
Quote:
In my opinion SSIM is very good metric (best I see until now) but sometimes it fails on : 1. Phsycovisual features like adaptive quantization. 2. Sometimes film grain and other miscellaneous details are considerated as artifacts and unwanted noise. So the video without noise but smooth and with some details lost has higher result. Maybe some new metrics are hard to implement apart to be slow but it's price of more powerful measure tool Last edited by IgorC; 3rd December 2006 at 01:51. |
|
3rd December 2006, 13:03 | #4 | Link |
Registered User
Join Date: Feb 2005
Location: London
Posts: 65
|
There is also "Brightness Independent PSNR"
http://www.compression.ru/video/qual...bi-psnr_en.htm which claims to be better than PSNR and SSIM. |
3rd December 2006, 19:37 | #6 | Link | |
aka XaS
Join Date: Jun 2005
Location: France
Posts: 1,122
|
Quote:
__________________
Q9300 OC @ 3.2ghz / Asus P5E3 / 4GB PC10600 / Geforce 8600 GTS |
|
3rd December 2006, 20:19 | #7 | Link | |
Registered User
Join Date: Apr 2004
Posts: 1,315
|
Quote:
I have some experience with SSIM values when it performs very well (and where OPSNR failed grossly) according to visual perception using adaptive quantization per macroblock in Nero H.264 codec. Dmitry Popov There are enough people who have alergy to PSNR family metrics. . Some fast review just show for now that BI-PSNR is just another mathematic PSNR metric *imo*. But I found MSU metrics like Blur and Blocking are usefull to undertsand where SSIM failed. For example, higher inloop filter in H.264 leads to higher SSIM values, higher blurness and lower blocking (according to MSU blur and deblock sets) however fow example in x264 : deblock 0 = it's balanced quality that gives more quality to low and middle freq. deblock -2 = it's another balanced quality that gives more quality to higher freq. details than to low/middle. So deblock values between (maybe) 0 and -2 are just part of personal preferences. But SSIM as it likes more blur than blocking will prefer higher deblocking. I had a sample where SSIM in/decreases and OPNSR de/increases when I change psy model. I will upload it when I'll have time. And will post some BI-PSNR values too. Last edited by IgorC; 3rd December 2006 at 20:31. |
|
3rd December 2006, 22:13 | #8 | Link |
AviSynth plugger
Join Date: Nov 2003
Location: Russia
Posts: 2,183
|
bad thing with most metrix (for video) is neglect of temporal effects.
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick I usually do not provide a technical support in private messages. |
4th December 2006, 02:55 | #9 | Link | |
Registered User
Join Date: Apr 2002
Location: Germany
Posts: 5,391
|
Quote:
No question the 2nd one gets the better metrics. No question the first one looks better. Dare to say, the first one *is* better.
__________________
- We´re at the beginning of the end of mankind´s childhood - My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!) |
|
4th December 2006, 03:41 | #10 | Link | |
Registered User
Join Date: Apr 2004
Posts: 1,315
|
Quote:
Can you provide avscript for this purpose? Last edited by IgorC; 4th December 2006 at 03:43. |
|
4th December 2006, 14:34 | #11 | Link |
Registered User
Join Date: Apr 2002
Location: Germany
Posts: 5,391
|
Can't say, didn't try yet. But that doesn't matter, since it's only an oversimlistic example.
The point is temporal consistency. Some (bigger) errors might be acceptable if they're "always the same", although they will give the worse metrics, compared to some (smaller) errors that are fluctuating back-and-forth (or whatever) between adjacent frames. Temporal consistency isn't evaluated by metrics that calculate their numbers only per-frame. Think of the fabulous "crawling wall" problem ... a very small per-frame error for metrics, but a highly annoying artefact to watch at.
__________________
- We´re at the beginning of the end of mankind´s childhood - My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!) |
4th December 2006, 18:23 | #12 | Link |
Registered User
Join Date: Dec 2003
Location: MO, US
Posts: 999
|
Sticking with ssim, there have been some proposed modifications:
multi-scale ssim complex-wavelet ssim (cwssim) weighted subband cwssim (wcwssim) All of those are still based on image quality/similarity assessment. Therefore, they would be applied to video on a frame by frame basis. And as MfA stated before, they are all more complex to implement than plain ssim. IMO, applying image quality metrics on a per frame basis to evaluate video is a lot like using a per-pixel metric (mse) to evaluate images. There are many types of artifacts/degradations that can lead to the same mse for two frames but that have quite different effects on percieved quality. Likewise, there are many possible temporal artifacts that can result in the same overall ssim (or another image quality metric) score for the sequence as a whole but that have significant differences on quality as percieved by a person. |
|
|