PF Comparer - Automated SSIM

[Paukenfell]

PF Comparer (http://www.mpeg-vs-avi.de/down.htm) is a small tool to compare MPEG and AVI-files with the SSIM-Plugin for AviSynth.

You can use .d2v, .avi and .avs files.

It's only an alpha version, so there will be many bugs in it.
AviSynth must be installed and the "d2v"-files I used were made with the newest "DGIndex".

I have made some tests with PF Comparer to compare HC, QuEnc and so on.
They are on my website (http://www.mpeg-vs-avi.de) (not finished)

[danpos]

@Paukenfell

It seems interesting, but for who aren't german native (like me) will be hard to read and therefore to figure out the results ... :(

CYA!!

[Paukenfell]

I don't think that you have to be german to read numbers ;-)
A number is the only result you get when you use my prog.

If you meant something else please say what and I'll translate it to the english language.
(my Homepage will be in english but this takes time because it's not even finished in german)

[Fishman0919]

Intresting test I did using PF Comparer (0.1.2 alpha)... Something I did before but this make it a little easier... it is a test comparing the diff IDCT's to the Org D2V file to show the errors/diff's each idct has

IDCT SSIM Time
--------------------------------------------
1 32-bit MMX 99.63% 2:15
2 32-bit SSEMMX 99.63% 2:14
3 32-bit SSE2MMX 99.63% 2:14
4 64-bit Floating Point 99.88% 2:41
5 64-bit IEEE-1180 Reference 99.93% 2:52
6 32-bit SSEMMX (Skal) 99.67% 2:11
7 32-bit Simple MMX (XviD) 99.74% 2:12


All idct were compared with a simple script and DGIndex 1.40 rc5 from the movie "The Pacifier"
(I did 4 diff movies but all gave the same results)
and was the first 7000 frames... time is how fast each idct was
encoded with CCE Basic with an AMD 64 3700+ OC (about 4000+)....

LoadPlugin("C:\Test\DGDecode\DGDecode.dll")
mpeg2source("C:\Test\test.D2V",idct=?)
ConvertToYUY2()

and compared dirtectly to the Original D2V file

[Paukenfell]

I tested some MPEG2-Encoders with PF Comparer.

The source was the intro of "Stargate".
I encoded with 1500 kbps and 352x576 and tried to use the same settings (e.g. QuEnc and HC have a lot of different settings).

Result:


AutoQMatEnc 0.24b -- 85.37
HC 0.15 -- 84.85
FreeEnc 0.31 -- 84.22
QuEnc 0.6 -- 79.77

[SAPSTAR]

I tested some MPEG2-Encoders with PF Comparer.

The source was the intro of "Stargate".
I encoded with 1500 kbps and 352x576 and tried to use the same settings (e.g. QuEnc and HC have a lot of different settings).

Result:


AutoQMatEnc 0.24b -- 85.37
HC 0.15 -- 84.85
FreeEnc 0.31 -- 84.22
QuEnc 0.6 -- 79.77

Interesting results..... :goodpost:

[gigah72]

I tested some MPEG2-Encoders with PF Comparer.

The source was the intro of "Stargate".
I encoded with 1500 kbps and 352x576 and tried to use the same settings (e.g. QuEnc and HC have a lot of different settings).

Result:


AutoQMatEnc 0.24b -- 85.37
HC 0.15 -- 84.85
FreeEnc 0.31 -- 84.22
QuEnc 0.6 -- 79.77



would be nice to know more details about the settings you used for each encoder,
otherwise it's not meaningfull at all.
(äpfel mit birnen ...)

[Paukenfell]

The settings I used:


AutoQMatEnc (used eclcreate)

Matrix: MPEG Standard
DC: 9
maxbitrate: 4000
minbitrate: 200
2 pass
GOP: 15
progressive

HC (used HC_GUI)

Matrix: MPEG
profile: best
DC: 9
maxbitrate: 4000
2 pass
GOP: 15
scene change on
progressive

FreeEnc

Template: Default
maxbitrate: 4000
2 pass
progressive


QuEnc

2 pass vbr
high quality
GOP: 15
scene detection on
DC 9
maxbitrate 4000
progressive


Sorry, that I didn't do this before.

[Fishman0919]

The settings I used:


AutoQMatEnc (used eclcreate)

Matrix: MPEG Standard
DC: 9
maxbitrate: 4000
minbitrate: 200
2 pass
GOP: 15
progressive

HC (used HC_GUI)

Matrix: MPEG
profile: best
DC: 9
maxbitrate: 4000
2 pass
GOP: 15
scene change on
progressive

FreeEnc

Template: Default
maxbitrate: 4000
2 pass
progressive


QuEnc

2 pass vbr
high quality
GOP: 15
scene detection on
DC 9
maxbitrate 4000
progressive


Sorry, that I didn't do this before.

Me like that :cool:

[danpos]

@Paulkenfel

I did a try with the app and then I got this error:

http://img231.imageshack.us/img231/7553/imagem9xm.png (http://www.imageshack.us)

I've tried to download the .ocx file but no chance.

Any idea?

TIA,

[gigah72]

programm is somehow strange. it gives me inconsistent results.
i open several times the same files, doing a compare and i receive different ressults?!

EDIT, more info:
i open my avs in vdub-mod and export 1000frames as helixyv12 avi
then i open the encoded m2v in vdub-mod and export the same 1000frames as helixyv12 avi

now i open the org avi and the m2v avi and compare then, let's say result is 86,19
i close and open again pfcomapre, now the result is 86,22 or something else

EDIT 2:
same inconsistent results with huffyuv as lossless codec

[Paukenfell]

@Danpos

Try to download the Visual Basic 6 runtime files.
If this doesn't help, I have no idea (but I will search)

@gigah72

The differences of the results are not very big and they are not a fault of my prog. I noticed it before but it's the "fault" of the SSIM-plugin for AviSynth.
In the next version there will be an option I call "3-pass-viewer". Then the video will be played 3 times and the average will be the result.
I hope that the results will be constant with this option.

[gigah72]

@Paulkenfel

I did a try with the app and then I got this error:

http://img231.imageshack.us/img231/7553/imagem9xm.png (http://www.imageshack.us)

I've tried to download the .ocx file but no chance.

Any idea?

TIA,

download here:
http://www.cricksoft.com/support/getfile.asp?ID=DLD17
copy to pfcomparepath, open commandline and write:
regsvr32 mci32.ocx

[danpos]

@gigah72

THX by donwload link and tip. :goodpost:

CYA!

[NaN]

I want to add my results to Fishman0919's idct comparison: I tested the new Walken & IEEE1180 compliant version of Skal's idct (was recently submitted to Xvid cvs) with the following results

Xvid simple: SSIM: 99.50% Time: 1:23
new Skal: SSIM: 99,63% Time: 1:20

At least on my sample (3000 frames) the new idct performs excellently.

Did anyone already test this idct?

Cheers, NaN

PS: I used PF Compare 0.1.2a.

[Paukenfell]

I made a new version months ago, but I could not test it, because my PC and AVISynth "don't like each other any more"!

http://www.mpeg-vs-avi.de/down.htm

Here you can download and test it!
There will be many, many bugs in it, sry!

I added a (very small) menu to the program in which you can edit 2 settings!

[NaN]

Great!

However you should really include the mci32.ocx file.

Greets, NaN

Edit: the new version won't even start! I get runtime error 71 - drive not ready? Maybe the old mci32.ocx is causing this?

[Paukenfell]

I fixed the old version (or I hope so).
http://www.mpeg-vs-avi.de/down.htm

@NaN:
1. I included the mci32.ocx file
2. I think you can start it now because I fixed some bugs! It would be great if you could test it!

[Prodater64]

Error "Run-time error 53. File not found"

[Paukenfell]

I fixed this bug. I tested it on two PCs and on every HDD.
http://www.mpeg-vs-avi.de/downloads/PF%20Comparer_0_1_3_2_a.rar

But I noticed that the .d2v, .avs and .avi files must be in the same drive than PF Comparer.

I hope I can fix it in the future!
And sorry again for the last versions but I am not a very experience coder (this is my second prog)!

I really hope that you can start it now.

[Prodater64]

I really hope that you can start it now.

Error "Run-time error 53. File not found", now when press "compare" button.

(Maybe related with spaces in path?)

[Prodater64]

You must not include dgdecode.dll as it works only with its dgindex pair.
So the user should to copy his own dgdecode.dll in PF comparer program.

Edited: The previous issue is related to blank spaces in the path.
You could add text boxes to select start and end frames, as you use trim in your avs, it is easy to change 0,0 by %start,%end values.

[Paukenfell]

@Prodater64:
I tested it with many blank spaces in the path and it worked.
On my computer it only doesn't work when the .d2v-file is on another HDD.

The next version will contain start and end boxes, thx for this idea!
I will also write a "warning" to use the own dgdecode.dll!

[Prodater64]

@Prodater64:
I tested it with many blank spaces in the path and it worked.
On my computer it only doesn't work when the .d2v-file is on another HDD.

:(
I will test it again, sorry.


The next version will contain start and end boxes, thx for this idea!
I will also write a "warning" to use the own dgdecode.dll!

Your welcome.

[Paukenfell]

http://www.mpeg-vs-avi.de/down.htm

Version 0.1.4 alpha:

- You can now use MPEG-files (automatically decoded by DGIndex)
- Second Menu: Path of DGIndex.exe and DGDecode.dll and some options

[Prodater64]

Thank you.
Keep the good work.

[Prodater64]

2 buttons, in settings and DGIndex popups say Cancle instead of Cancel.

[Prodater64]

Many bugs.

1 - Error when in another disk than C:\.
2 - Error when choice a device with no disk.
3 - It seems still error when long paths with spaces. (check it again, please)
4 - AVS error when in C:\: LoadPlugin: unable to load "C:\"


# **made by Paukenfell**

LoadPlugIn("SSIM.dll")
LoadPlugIn("C:\")
LoadPlugIn("Simpleresize.dll")

source=AVIsource("C:\temp\a.avi").ConvertToYV12().SimpleResize(720,576).trim(1,50)
encoded=AVIsource("C:\temp\b.avi").ConvertToYV12().SimpleResize(720,576).trim(1,50)

SSIM(source,encoded,"C:\temp\b.avi.csv","C:\temp\b.avi.txt",lumimask=false)

[Paukenfell]

Thanks for testing !

I will have time this weekend to search for solutions for this bugs.

Bug 4 - You must load the full path (C:\DGDecode.dll) !

Bug 1 - Can you please check if there is an error when loading the source-file from another disk and the encoded file from Disk C.

[Prodater64]

Thanks for testing !

I will have time this weekend to search for solutions for this bugs.

Bug 4 - You must load the full path (C:\DGDecode.dll) !

Bug 1 - Can you please check if there is an error when loading the source-file from another disk and the encoded file from Disk C.

1 - "File not found" error.

and more:

5 - When you press any of both load buttons, and dialog popups, menu items and other buttons in main form are not disabled. If you press any of them an error popups and program crash.
6 - Buttons named Cancle instead of Cancel.

[Paukenfell]

http://www.mpeg-vs-avi.de/down.htm (v. 0.1.4.1 a)

@Prodater64:

1 and 3 - I tested it in "H:\test sd sadf as dds sdaf kf sad sdak fsda ksdaf kdaf kadf kda sd\sdahfuahr auserh uear sa fsaiuerz ar a i is isur ur isaurir uieur iruieru\blaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaa aaaaaaaaaa aaaaaaa" and it worked! Please test it!

2 - fixed
4 - now has C:\DGDecode.dll in it
5 - now you don't see the other forms when you press a button -> You can't press any button
6 - Buttons are now called Cancel

Thanks for testing! I hope that the prob with long pathes is now solved!

[Prodater64]

1 - It don't work in any folder in HD other than C:\ (PF Comparer in same or not, folder than the compared files)
2 - When select "compare all" the following lines are generated in avs file:

source=AVIsource("C:\PFComp\a.avi").ConvertToYV12().SimpleResize(720,576).trim(False,50)
encoded=AVIsource("C:\PFComp\b.avi").ConvertToYV12().SimpleResize(720,576).trim(False,50)


3 - When in D:\ HD (not cd neither dvd drives), error and crash.

[Paukenfell]

Sorry for not replying to this bug but in the last months I had 3 different PC (CPU didn't want to work any longer but that's life).

I had a new version of PF Comparer which was lost with my second computer!

Actually I'm working (again) on a new version which will have less bugs (I hope so :) ).
I'll post when I'm finished (won't take so much time because I only have to add some extras)

[Paukenfell]

PF Comparer 0.2 beta (http://www.mpeg-vs-avi.de) is now finished!

Changelog:

~ made a new prog
~ video is now integrated
~ resize-filter can be chosen
~ changed the "Open-File"-Dialog and hope that this one will work
~ Resolution can now be set ("Resize Filter" must be activated)

Microsoft .net 2.0 must be installed!!!

The open-file bug is fixed because I use the "Common-Dialog-Control" which works better then my own open-File-Dialog.

Please post all bugs you find (Hope that I found the most)