Video quality assessment

[koliva]

Hi,

My question is rather long but I know that some of you have just 5 minutes to shed a light on this issue.



Thank you very much.

[fvisagie]

Although subjective review is always desirable (in my opinion), if you want to (also) do an objective comparison you could use something like Avisynth's Compare():

This filter compares the original clip clip_original and its filtered version clip_filtered. The filtered version will be returned with the results of the comparison. Per frame the Mean Absolute Difference, Mean Difference and Peak signal-to-noise ratio (PSNR) is given, as well as the min (minimum), avg (average) and max (maximum) PSNR up to that frame (calculated frame-wise). Starting from v2.53, the 'Overall PSNR' (calculated over all pixels in all frames) is also shown on the output clip.

[Dark Shikari]

I'm not sure what the purpose of this would be; wouldn't the results from this test already be rather expected? ultrafast is going to be pretty awful even compared to superfast.

[Selur]

The audience will always compare the right screen against the left screen.
it might also be a good idea to show the audience the source image, otherwise when reading 'which video do you think is more flawless' the question arises 'How to say if a reencode is flawless if I don't know the source?'

[koliva]

Although subjective review is always desirable (in my opinion), if you want to (also) do an objective comparison you could use something like Avisynth's Compare():

For this, I guess I need to store the decoded file right? If so it is not so applicable in my application. Actually it is applicable but maybe after finishing this test first.

it might also be a good idea to show the audience the source image, otherwise when reading 'which video do you think is more flawless' the question arises 'How to say if a reencode is flawless if I don't know the source?'

That's correct. The audience will see the sources at the same time as well. Thanks.

I'm not sure what the purpose of this would be; wouldn't the results from this test already be rather expected? ultrafast is going to be pretty awful even compared to superfast.

To me, the audience is going to select the second one. The reason is that the frame rate plays a crucial role in the "quality" understanding and the second one produces quite low bitrate & high frame rate. Since the encoded video is transmitted over the wireless network, frame rate differs a lot between first, second and the third codec. First codec gives nice framerate result but also generates a high bitrate (lower than the limit but still high.). Second one produces slightly lower framerate but above 20 fps most of the time and a lot better bitrate compared to the first one. The third one is also good but the frame rate is rather low and recognizable.

By the way, I forgot mentioning that my application runs real-time. So the zerolatency is very important to a certain level.

[Dark Shikari]

A "video quality test" in which the bitrates (and frame rates!) differ between videos doesn't make any sense. You need to select a single independent variable; multiple independent variables make it impossible to make any real conclusions.

[koliva]

A "video quality test" in which the bitrates (and frame rates!) differ between videos doesn't make any sense. You need to select a single independent variable; multiple independent variables make it impossible to make any real conclusions.

You are right but my user cases are very broad from watching HD movie to having a presentation on the screen. I need to find an "optimum" setting covering all extreme cases. For this purpose, I use VBV encoding. Basically I am tying to find a codec with proper settings for the worst case which is HD video with high motion and therefore it will already be a nice codec for best cases like presentation and so on.