Program to Compare Video Encode Quality [Archive]

lexor

23rd July 2008, 17:33

try programming your eyes.
I don't get it, where is the funny?

As per OP's question, Avisynth can let you stack frames from multiple videos and have them play back simultaneously in player like MPC. Check out the avisynth docs and sub-forum.

Inventive Software

23rd July 2008, 17:38

try programming your eyes.

:stupid:

EDIT: Cos I apparantly violated the rules (fair cop BTW, not complaining), I'll elaborate. Only your eyes can tell the difference between videos. Metric comparisons can only go so far, and in my experience are wholly unreliable.

LoRd_MuldeR

23rd July 2008, 18:15

The question is whether the OP wants a tool that helps the user to compare videos visually or a program that will do the comparison automatically/mathematically without the help of human eyes ;)

Of course the latter is not possible for most purposes! And I think that is what Sharktooth meant...

setarip_old

23rd July 2008, 18:27

@Sem

Hi!

I'd suggest to you that "video Quality" is a subjective concept "in the eye of the beholder", because different viewers have different likes and dislikes regarding video - e.g. some prefer a softer image, some a sharper image, some prefer brighter colors, while others prefer "mellower" colors...

Sharktooth

23rd July 2008, 19:14

for who didnt get it... read the title again. only EYES can compare quality...

LoRd_MuldeR

23rd July 2008, 19:17

for who didnt get it... read the title again. only EYES can compare quality...

Yes, but there are programs to that can help your eyes. Like inspecting the video frame-by-frame and switching between different versions/encodes easily...

Sharktooth

23rd July 2008, 19:20

he asked for a "Program to Compare Video Encode Quality"
it does NOT exist. there are program based on commonly used quality metrics, but we all know they are not good for REAL quality comparison.
Infact x264 psy-RDO is crap for those programs, while, in reality, it enhances the perceved quality a LOT.
Only the eyes are good...

LoRd_MuldeR

23rd July 2008, 19:32

he asked for a "Program to Compare Video Encode Quality"

And that question is not clear....

"Program to Compare Video Encode Quality" might also refer to a tool that shows the same frame from different encodes side-by-side or alternating, so you can compare the encodes more easy ;)

Sharktooth

23rd July 2008, 20:04

he asked for something and not some other thing.
it's CLEAR he wants a PROGRAM that COMPARES QUALITY of different encodes... not a program that helps him to judge quality of different encodes. it doesnt exist. PERIOD.

LoRd_MuldeR

23rd July 2008, 20:13

he asked for something and not some other thing.
it's CLEAR he wants a PROGRAM that COMPARES QUALITY of different encodes... not a program that helps him to judje quality of different encodes.

IMHO that is not so clear. So we will have to wait for his answer... :p

Sharktooth

23rd July 2008, 20:16

if he was looking for something different he could have used different word or the search function and would have found avisynth...

Sem

23rd July 2008, 20:50

And that question is not clear....

"Program to Compare Video Encode Quality" might also refer to a tool that shows the same frame from different encodes side-by-side or alternating, so you can compare the encodes more easy ;)

maybe i should have worded my post better but thats pretty much what i would like

but its ok now what i will do is make a copy of my mplayerc.exe somewhere and have players open playing at the same time and then compare the 2 encodes that way

thanks for all the reply's tho

LoRd_MuldeR

23rd July 2008, 20:55

Try this in Avisynth:

clip1 = DirectShowSource("C:\MyEncode1.foo", audio=false)
clip2 = DirectShowSource("C:\MyEncode2.foo", audio=false)

StackVertical(clip1,clip2)

Poopoo

23rd July 2008, 21:02

Thanks for the solution Mulder,

too bad some guys" Ego's increase with their postcount.:rolleyes:

Sem

23rd July 2008, 21:05

Try this in Avisynth:

clip1 = DirectShowSource("C:\MyEncode1.foo", audio=false)
clip2 = DirectShowSource("C:\MyEncode2.foo", audio=false)

StackVertical(clip1,clip2)

nice one :D that works brilliantly much better than having 2 instances of MPC open

i havent tried it yet with my 720p and higher encodes as i doubt i will have enough screen space

but for my dvd encodes this works fine

thanks again :thanks:

LoRd_MuldeR

23rd July 2008, 21:07

i havent tried it yet with my 720p and higher encodes as i doubt i will have enough screen space

If it doesn't fit on your screen, stack horizontal and use "extended desktop" on two screens ;)

Zwitterion

23rd July 2008, 21:43

LoRd_MuldeR

23rd July 2008, 21:49

You could also try this tool here:
http://forum.doom9.org/showthread.php?t=138430

It lets you compare two videos in fullscreen switching between them with the left mouse button. For me it's easier than side by side comparing ala stackhorizontal/-vertical.

You could achieve that with Avisynth like this:
clip1 = DirectShowSource("C:\MyEncode1.foo", audio=false)
clip2 = DirectShowSource("C:\MyEncode2.foo", audio=false)

Interleave(clip1,clip2)

rack04

23rd July 2008, 22:04

Try this in Avisynth:

clip1 = DirectShowSource("C:\MyEncode1.foo", audio=false)
clip2 = DirectShowSource("C:\MyEncode2.foo", audio=false)

StackVertical(clip1,clip2)

Just to add on this script. I always add a subtitle so that I know what I'm comparing.

For example:

source1 = DirectShowSource("C:\Personal\Videos\Van Helsing DXVA-HD-FAST.mp4",fps=23.9759856527702,audio=false)
source2 = DirectShowSource("C:\Personal\Videos\Van Helsing SA-HD-DVD.mp4",fps=23.9759856527702,audio=false)

StackVertical(source1.subtitle("DXVA-HD-FAST"),source2.subtitle("SA-HD-DVD"))

Avenger007

23rd July 2008, 23:46

Is it just me or does anyone else notice MPC uses TV (16-235) color levels for playing avs files instead of PC (0-255) colors? This results in the quality of the images being lower than they actually are.
However, MPC uses PC (0-255) for all media files, so why not for avs files? :confused:

Ranguvar

24th July 2008, 01:25

Avenger007

24th July 2008, 01:43

If MPC uses a renderer other than Haali's or the Overlay Mixer, if the decoder is outputting non-RGB video, the color levels will be limited.

Either use one of those renderers, or force ffdshow to output RGB32/RGB24 video only, with high quality YV12 -> RGB conversion enabled, and the RGB Conversion set to Full Range.
If that was the case then media files (avi, wmv, mp4,..) would use TV levels to.
BTW, both Haali's and the Overlay Mixer also cause media files to use TV levels, so I'm using VMR9 (renderless). Also, all media files and avs files use PC levels in Windows Media Player.

Can you or anyone else confirm my observations? :confused:

Sharktooth

24th July 2008, 02:36

it's all ok here.

Sharktooth

24th July 2008, 02:44

Thanks for the solution Mulder,

too bad some guys" Ego's increase with their postcount.:rolleyes:
better speak for yourself.
as this is a public forum, everyone can reply with a valid answer.
what the original poster asked in the first post does not exist but since he was looking for some other thing instead (his request was just badly worded) he got some valid replies from others.
now, if you're finished with insults/flames, since now we all know what the original poster wants, we can help him.

Guest

24th July 2008, 02:51

Gents, please stay on topic. Any further gratuitous ad hominems will lead to strikes (attention: Poopoo!). Thank you.

Avenger007

24th July 2008, 03:08

it's all ok here.
Thanks for the response.
I created a thread with the problem here: http://forum.doom9.org/showthread.php?t=139623

WRT the topic, I use MPC to compare frames by removing the status and menu bars using ctrl+1, ctrl+2, ctrl+5 and ctrl+0 so the image fills the entire window. :)

vmrsss

27th July 2008, 12:21

Zarxrax

27th July 2008, 17:38

I like to use AvsP for comparing clips. It's tabbed interface lets you easily load up a lot of avisynth scripts and then change between them using the number keys. Very useful if you have more than 2 clips to compare.

Sharktooth

28th July 2008, 02:12

Hi everybody. I dont think the original question in ill-posed, at the outset only eyes can compare quality, but I'd say that one of the ideas is to identify better and better metrics that approximate that comparison; otherwise most of the developments described in this forum would be pointless...
metrics != quality.

metrics like PSNR, SSIM, VQF, etc measure the difference from the source. there is some relation with quality but a higher value does not always mean higher quality since the human eye has a very complex perception that cant be just described as a mathematical function.

Esurnir

29th July 2008, 00:39

clip1 = DirectShowSource("C:\Foo.xxx", audio=false)
clip2 = DirectShowSource("C:\Bar.yyy", audio=false)

StackHorizontal(clip1.crop(0,0,-320,0),clip2.crop(320,0,0,0))
I personnally like this one, the difference are more striking that way in my opinion.

LoRd_MuldeR

29th July 2008, 01:10

My attempt of a simplistic tool to compare encodes:
http://www.mediafire.com/?bhkl0qhknne

Put it in your VirtualDub folder and make sure Avisynth is installed ;)

Ytterbium

5th August 2008, 09:11

This seemed quite good, around 94% seemed good with my eyes.

http://www.ece.uwaterloo.ca/~z70wang/research/ssim/

and again in MatLab

http://www.mathworks.com/access/helpdesk/help/toolbox/vipblks/index.html?/access/helpdesk/help/toolbox/vipblks/ref/psnr.html

Sharktooth

5th August 2008, 17:09

read my previous post. metrics do not mean quality.
infact psychovisual enhancements usually lower the metrics but rise the perceived quality by much.
you cant automatically compare quality with a program or using metrics... you still need your eyes.

poisondeathray

5th August 2008, 18:12

I usually use my eyes to compare for the reasons Sharktooth gave

I've noticed a trend when taking a series of frameshots for comparison from the same frames (between 2 different encoding settings from same source, same bitrate), when converting the .bmp => .png (or even a direct .png capture), the "higher quality encoded" frame is almost always larger in size (Disc space). Of course the .bmp screenshot between the 2 encodes is the same size. I'm guessing that the "higher quality" frame has more detail retention and information in that frame, and this is an objective metric. I'm not sure what goes into the .png compression algorithm - it might be similar to the PSNR, SSIM measurements?

Sharktooth

5th August 2008, 18:14

no, but it can be a false measure since compression algos prefer certain data sequences instead of others...

Ytterbium

10th August 2008, 00:06

I agree that your eye are the best tool, It pointless in compairing 2 lossey picture files as you wouldn't be measuring the loss from codec. I think PNG is lossless, it uses compression like zip which is why png's are bigger than jpg's. bmp is certinally lossless as it's pure uncompressed data.

I found that you can do PNSR (Compare), SSIM & VqmCalc in Avisynth. SSIM is supposed to be much closer to human perception that PNSR.

ggf31416

10th August 2008, 14:07

the "higher quality encoded" frame is almost always larger in size (Disc space). Of course the .bmp screenshot between the 2 encodes is the same size. I'm guessing that the "higher quality" frame has more detail retention and information in that frame, and this is an objective metric.

by that "metric" a codec that outputs pure white noise gives the highest quality :scared:

henryho_hk

12th August 2008, 02:41

Considering the "performance" of common TFT LCD screen, I don't think it is a good idea to stack the video horizontally or vertically. Even the same video looks different on different position of the screen. The distorted color, brightness, contrast, etc. have a great effect on the perceived quality.

Ranguvar

12th August 2008, 04:11

Yeah, I personally dislike stacking, and much prefer using something like AvsP to flip between the same frame of each encode back and forth. Differences are much more visible this way.

LoRd_MuldeR

14th March 2013, 22:24

My attempt of a simplistic tool to compare encodes:
http://www.mediafire.com/?bhkl0qhknne

Put it in your VirtualDub folder and make sure Avisynth is installed ;)

Attached a slightly updated version on user request.