How to evaluate the quality of Video files??

[carmiyu]

Hi,

Yesterday, my friend and i went to buy a new cellular phone. There were three cellular phones my friend was
considering: Nokia, Sony Ericsson and Samsung.
Video recording quality is one of the key item to my friend.
When she looked around, she asked a question that
startled me : is there any tool or equipment which
can evaluate the video quality modeling the behavior of
human eyes??

What a question!! I never thought about that before.
Although everybody may have their own preference
on video/image quality, is there a tool for engineers
developing video encoder to evaluate their video encoder
quality?? How do they know their encoder algorithms,
such as rate control algorithm, give good quality??? I guess
if there is such a tool, this tool must truly model the
behavior and preference of human eyes.


Thanks for any feedback.

Carmiyu

[CWR03]

Short answer: no. Something like that is still far ahead of our technical ability. Even if it were, perception is different with every person. One person may not care as much about the detail of an image as might another.

[Blue_MiSfit]

IMHO, beyond the obvious, video quality is totally subjective.

Everyone can agree that really old compressed video looks simply dreadful. Anyone remember Wing Commander 3 with its multiple CDs of blockfest MS Video 1 in DOS? How about old Cinepak stuff on mac? The video clips in Myst? :) Obviously, DVD quality MPEG-2 is light years better to this, to almost anyone that looks at it.

That said, many people (believe it or not) either dont see or dont care about the difference between for example an old 1cd DivX 3.11 rip of The Matrix, and a brand spanking new x264 rip. To us experienced professionals, the difference is obvious and shocking. To 90% of people, the difference is barely noticeable. I have sent 150kbit WMV clips to friends and had them say "wow thats great quality", when they were seeing a smudged mess of garbage.

Metrics like SSIM and PSNR give you a vague idea of "codec a is better than codec b", but for the most part they're too objective, and dont take into account how the eye works. Thats why things like custom matricies, which are based on studies of the human visual system (HVS) usually score lower on metrics than default matricies, but look better to many who use them. However, to some, these custom matricies do not look as good. These perceptions can be subconsciously based on what we believe to be true IMHO.

There's no way to know for sure. You've just gotta try :)

~MiSfit

[foxyshadis]

MSU is working on research in that direction though. Absolute quality measures, as opposed to nearness to an "ideal" picture. But human preferences are too important a variable to ignore. =p Measurement categories could be scaled based on personal irritation factors though.

[mod]

Anyone remember Wing Commander 3 with its multiple CDs of blockfest MS Video 1 in DOS?
Me.. terrible.
BTW, I agree, for a lot of ppl if you press play and it plays, well, it's perfect quality :p