Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 AVC / H.264

Reply
 
Thread Tools Search this Thread Display Modes
Old 15th November 2012, 10:31   #1  |  Link
ff58
Registered User
 
Join Date: Sep 2012
Posts: 6
H.264 encoder evaluation tool

Hi,

I want to see the performance of our h.264 encoder which is running on an embedded system. For this purpose, I have encoded a raw YUV file and obtained a compressed h.264 file. I want to ask if it is possible to compare the YUV and h.264 files in terms of related measures such as PSNR? Is there any software tools than can be capable of doing these kind of comparison measurements?

Any other evaluation methods and tools will be very helpful?

Thanks&Best Regards,
Fikret
ff58 is offline   Reply With Quote
Old 15th November 2012, 17:59   #2  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,239
x264 can calculate both, PSNR and SSIM, by using "--psnr" and/or "--ssim". If you use those, you should also set "--preset psnr" or "--preset ssim", respectively.

As for other H.264 encoders, I don't know if they have a similar feature, but you may always use the MSU tool:
http://compression.ru/video/quality_...t_tool_en.html

Having said that, you should be aware that objective quality metrics, such as PSNR or SSIM are not very good to judge the overall quality

(We all know that the "Psy" optimizations in x264 can give a great perceived quality improvement, still they significantly hurt the PSNR and SSIM metrics)
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊
LoRd_MuldeR is offline   Reply With Quote
Old 16th November 2012, 10:06   #3  |  Link
ff58
Registered User
 
Join Date: Sep 2012
Posts: 6
Quote:
Originally Posted by LoRd_MuldeR View Post
x264 can calculate both, PSNR and SSIM, by using "--psnr" and/or "--ssim". If you use those, you should also set "--preset psnr" or "--preset ssim", respectively.

As for other H.264 encoders, I don't know if they have a similar feature, but you may always use the MSU tool:
http://compression.ru/video/quality_...t_tool_en.html

Having said that, you should be aware that objective quality metrics, such as PSNR or SSIM are not very good to judge the overall quality

(We all know that the "Psy" optimizations in x264 can give a great perceived quality improvement, still they significantly hurt the PSNR and SSIM metrics)
Dear Lord_Mulder,

Thanks for your reply. If PSNR and SSIM are not very good, is there any other tools for evaluating the quality?

I want to see the effect of changing encoder parameters and determine the best configuration. So, you say that the best way to evaluate the quality is the subjective evaluation?
ff58 is offline   Reply With Quote
Old 16th November 2012, 12:04   #4  |  Link
pandy
Registered User
 
Join Date: Mar 2006
Posts: 1,049
Quote:
Originally Posted by ff58 View Post
Thanks for your reply. If PSNR and SSIM are not very good, is there any other tools for evaluating the quality?
Yes, human brain + eye (simulator?) however such solutions are very expensive.

I think You misunderstood Lord Mulder - PSNR or SSIM are perfect tools to measure PSNR or SSIM however there is no direct translation for PSNR or SSIM to perceived - subjective quality - so PSNR or SSIM are good tools to compare PSNR or SSIM however best PSNR or SSIM encoder can be not even close to perceived, subjective quality.
pandy is offline   Reply With Quote
Old 16th November 2012, 13:20   #5  |  Link
ff58
Registered User
 
Join Date: Sep 2012
Posts: 6
Quote:
Originally Posted by pandy View Post
Yes, human brain + eye (simulator?) however such solutions are very expensive.

I think You misunderstood Lord Mulder - PSNR or SSIM are perfect tools to measure PSNR or SSIM however there is no direct translation for PSNR or SSIM to perceived - subjective quality - so PSNR or SSIM are good tools to compare PSNR or SSIM however best PSNR or SSIM encoder can be not even close to perceived, subjective quality.
Dear Pandy,

Ok, I understand now. I have tried MSU software, it is really good for comparison. Thank you for your helps.
ff58 is offline   Reply With Quote
Old 16th November 2012, 17:33   #6  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,239
Quote:
Originally Posted by ff58 View Post
I want to see the effect of changing encoder parameters and determine the best configuration. So, you say that the best way to evaluate the quality is the subjective evaluation?
Yes, definitely.

There are many examples where objective metrics (like PSNR or SSIM) can give a misleading results. As mentioned before, the "Psy" optimizations in x264 are known to give a huge perceived quality boost, but if you only trusted on PSNR or SSIM, you would think they reduce the quality. What you can learn from this is that objective metrics can be helpful for some tasks (sometimes you simply have to get "hard" numbers), but you should never blindly trust them. Basically, all commonly used metrics measure the "error" between the original and the encoded video and then express "quality" as the absence of error (assuming "perfect" quality for the original). That works up to some point, but it's not how the human eye works. Sometimes more "error" looks better than smoothing all the details out...
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊

Last edited by LoRd_MuldeR; 16th November 2012 at 17:47.
LoRd_MuldeR is offline   Reply With Quote
Old 17th November 2012, 06:59   #7  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,957
The problem with subjective comparison (in my experience) is that we're easily confused, and the differences are often quite minor - especially when making small changes across a large sample. Placebo effect also comes into play.

I've yet to find a foolproof way to compare videos.
__________________
These are all my personal statements, not those of my employer :)
Blue_MiSfit is offline   Reply With Quote
Reply

Tags
evaluation, h.264 encoder, performance, test

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 10:35.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.