SSIM 0.24, an objective video quality metric

[Junior73Italy]

@GannJunior
Hello Gann is Junior73 of doom9.it The problem interests also me
because is creating with to Robot1 an utility called AUTOSSIM that
automates the calculation of the SSIm working with DGindex, Virtual
dub mod and avisynth.The program we must still publish .The first version will be beta1,I hope. Here the link of the Italian forum for greater info.:

http://forum.doom9.it/viewtopic.php?t=8484

We will add to the program new features in the future. I have one
question to make to Lefungus on the direct comparison of two field between two interlaced clip.But my English is not very well.

Can you help me?

The Avisynth script I use :

LoadPlugin("C:\Programmi\AviSynth 2.5\plugins\SSIM.dll")
LoadPlugin("C:\Programmi\AviSynth 2.5\plugins\DGdecode.dll")
A=DirectShowSource("G:\CCE Prove\Clip dv.avi",25,audio=false,video=true).SeparateFields().SelectEven().ConvertToYV12()
B=DirectShowSource("G:\CCE Prove\Clip dv.avi",25,audio=false,video=true).SeparateFields().Selectodd().ConvertToYV12()
C=mpeg2source("G:\CCe Prove\Tmpegdv 5000.d2v").SeparateFields().SelectEven().ConvertToYV12()
D=mpeg2source("G:\CCe Prove\Tmpegdv 5000.d2v").SeparateFields().Selectodd().ConvertToYV12()
Semiquadro1=SSIM(a,c,"semiquadro1.csv","semiquadro1average.txt",lumimask=true,scaled=false)
Semiquadro2=SSIM(a,d,"semiquadro2.csv","semiquadro2average.txt",lumimask=true,scaled=false)
StackVertical(semiquadro1,semiquadro2)

GoodBye

@Wilbert

Sorry....

[Wilbert]

@Junior73Italy,

This is an english forum. I hope you can add an english summary/translation to your post.

[Gannjunior]

Hi,
i'm going on the beach now...:D
Manao, tonight I'll post my scripts ;)
Willbert, in previous post Junior73 has asked me to translate his question for Lefungus; i'll do it tonight ;)

[OT mode on]
Ciao Junior, poi vengo a leggere bene il post su doom9italy per comprendere bene il problema:)
[OT mode off]

ciaoo!

[Gannjunior]

Manao, this is the simple encoding script used for all my tests on 7th chapter of blade runner.

LoadPlugin("C:\AVS\Plugin\AviSynthPlugins\dgdecode.dll")
#
mpeg2source("F:\Burner_Temp\Blade Runner\c7.d2v")
crop(30,86,656,410)
trim(0,1700)
LanczosResize(640,272)


and this is the SSIM script.

SetWorkingDir("D:\Test\ExtraPlugins\AviSynthPlugins\")
LoadPlugin("DGDecode.dll")
LoadPlugin("ssim.dll")

source=mpeg2source("F:\Burner_Temp\Blade Runner\c7.d2v").crop(30,86,656,410).LanczosResize(640,272).trim(0,1700)
encoded=aviSource("D:\Test\5.20\207.avi")
return ssim(source,encoded,"D:\Test\res.csv","D:\Test\avgSSIM.txt",lumimask=true)


Obviously, every time I needed, i modified it to work correctly for xvid + bf etc...I haven't any problem, except to calculate SSIM in divx 5.2 + multi bf test where I obtain illogical values...
Ans I'm sure to compare exactly the same frames between source and encoded file, 'cause I've checked them frames after frames...

http://gannjunior.altervista.org/immagini/doom9/test.GIF


In another test i'm doing on a different source (4th vob of pearl harbor), in presence of multi bf, i can't achieve values different from 3.71 or 3.78%, even if i change bitrate...(with resulting changement of size...).
So, what i've explained, is my problem with divx 5.2... (besides to the fact mpeg2 matrix give me saturation size by far smaller than h263...:mad: ...bug?!)

@Wilbert
Later I'll get all informations from Junior73 about his question for Lefungus and i'll post it for him.

ciauzz!:)

[bgpop]

hi, gannjunior... your script seem correct...
They are something of strange on the b-frames implementation of divx 5.2 (also assuming that Xvid's multipls b-frames don't cause any trouble).
I think i will do some tests to verify ur problem...


ciao bello ;)

[Manao]

http://home.student.utwente.nl/m.f.al/ssim/

Since you posted on that thread, you should have got the latest version. BTW, thank you, it allowed me to see that my SSIM version wasn't up to date ( I was having the same issue as you were ).

[MfA]

Buffer overflows do funky things ...

[HLW]

Hi!

I wonder, is it possible in any way to run several ssim-analyses in a row on different targets (for instance with different bit rates) without having to run the script over and over again, and then make the ssim-results be written into different files?

- Haakon

[redfordxx]

When I have the .csv file with SSIM and weight for each frame, how can i calculate SSIM average for some part of video?

[On2Tech]

I am trying to understand the ssim code as it stands now.

I am particularly interested, in the usage of the luma mask.

I think the concept behind luma mask option is that when there are mixed brightness pixels in the same image bright pixels are more important than dark pixels. I think that there is some merit to this point, and even some research to back it up.

However the way the code is written ssim extends that concept beyond images to entire frames. In affect in this test dark frames are weighted far lower than light frames.

Imagine a video clip in which the first half of the clip includes a dark frame with white text in one small section of the screen, and the second half of the clip involves the exact opposite black text on a mostly bright frame.

The second half of the clip could be given a weighting nearly 100 times the first half of the clip.

I don't think most people would find a compression of these clips that threw 100 times as many bits on the second half than the first half more appealing than one that gave both halves of the clip roughly the same number of bits.

As such I suggest a slight alteration to the code.

I propose a change from:

d = (0.8f*a)+(0.1*(b+c));
summedWeights += frameWeight;
summedQuality += d * frameWeight;

to

d = (0.8f*a)+(0.1*(b+c));
summedWeights += 1;
summedQuality += d;

this avoids the issue by using luma masking to determine quality of each given frame, but then just averages this value across the wide spectrum of frames.

[dr.Prozac]

Hi !
I have just written the following script and it doesn't work at all (MPC and VDMod crash when open the script).
I compare XviD clip to oryginal dvd project. My encoded avi consists of 159845 frames (I used trim during encoding) and this is the reason of trim(...,159845) in my script. I decided to use trim(3,...) because of using XviD with B-frames.


LoadPlugin("C:\PROGRA~1\GORDIA~1\DGMPGDec\DGDecode.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\SSIM.dll")

a=mpeg2source("D:\My documents\My video\Apartament\Apartament.d2v").trim(3,159845).crop(2,12,716,552).lanczosresize(544,288)

b=AVISource("D:\My documents\My video\Apartament\Apartament.avi").trim(3,0)

return ssim(a,b,"d:\results.csv","d:\averageSSIM.txt",lumimask=true)


Could you please suggest me what is wrong with this script (?)

[redfordxx]

Originally posted by dr.Prozac
Could you please suggest me what is wrong with this script (?)
Try BSPlayer --- craches rarely, sometimes ends with excepion but does the work. (got WinXP SP2)

[Lefungus]

Originally posted by On2Tech
I am trying to understand the ssim code as it stands now.
[...]
As such I suggest a slight alteration to the code.

I propose a change from:

d = (0.8f*a)+(0.1*(b+c));
summedWeights += frameWeight;
summedQuality += d * frameWeight;

to

d = (0.8f*a)+(0.1*(b+c));
summedWeights += 1;
summedQuality += d;

this avoids the issue by using luma masking to determine quality of each given frame, but then just averages this value across the wide spectrum of frames.

Well, SSIM is just implemented the way it is demonstrated in the white papers and especially in Video Quality Assessment Based on SSIM (http://www.cns.nyu.edu/~zwang/files/papers/vssim.pdf)
It could surely be improved, like adding the motion weight, coupled with the Multi-scale index.
About modifying the overall value calculation, well, I would prefer sticking to the paper since it actually has some testing behind even if it could be seen as inappropriate.

[dr.Prozac]

Thx redfordxx. Unfortunately BSPlayer also crashes on this script. Something incorrect must be inside it.

[redfordxx]

Originally posted by dr.Prozac
Thx redfordxx. Unfortunately BSPlayer also crashes on this script. Something incorrect must be inside it. Yeah, I was afraid...

Try another thing:
1. Make source.avs which equals your avi (e.g. checke in VDMod that it starts and ends with same frame --- no bframe lag, same resolution)
2. write script like
a=AVISource(...source.avs).trim(100,200)
b=AVISource(...avi).trim(100,200)
SSIM...
3. If it works, make it longer --- if not, start cropping a and b.
Mostly it crashed, for me helped to crop 8,16 or so borders. Don't know why but helped...

[dr.Prozac]

You were right redfordxx. Tht was my mistake - I didn't count the number of frames properly. Just one frame missing at the end of the clip. So I think it's a good idea to trim as you've suggested to be safe and protected against the difference of the number of frames.
I just wonder why the value of Average SSIM is > 1 in the txt file.
I got a result Average SSIM= 82.09 (?) It looks like a sum - not average.

[redfordxx]

Originally posted by dr.Prozac
You were right redfordxx. Tht was my mistake - I didn't count the number of frames properly. Just one frame missing at the end of the clip. So I think it's a good idea to trim as you've suggested to be safe and protected against the difference of the number of frames.
I just wonder why the value of Average SSIM is > 1 in the txt file.
I got a result Average SSIM= 82.09 (?) It looks like a sum - not average.
It's OK. I guess Manao wrote somewhere something about how's that calculated. Whatever, just take it. Full quality=100(%)

[dr.Prozac]

It's OK. I guess Manao wrote somewhere something about how's that calculated. Whatever, just take it. Full quality=100(%)
I see :)
Thanks for yor posts redfordxx.

[Lefungus]

Originally posted by redfordxx
It's OK. I guess Manao wrote somewhere something about how's that calculated. Whatever, just take it. Full quality=100(%)

Or you can read the first page of this thread where I explain what those values mean...
It can be disabled if you do prefer "unscaled" results.

[On2Tech]

Originally posted by Lefungus
Well, SSIM is just implemented the way it is demonstrated in the white papers and especially in Video Quality Assessment Based on SSIM (http://www.cns.nyu.edu/~zwang/files/papers/vssim.pdf)
It could surely be improved, like adding the motion weight, coupled with the Multi-scale index.
About modifying the overall value calculation, well, I would prefer sticking to the paper since it actually has some testing behind even if it could be seen as inappropriate.

I understand. Thanks for the reply and for the useful plugin. Seemed odd to me but perhaps that's the way it was intentioned.

[Poutnik]

Can I get SSIM value of compared 2 frames by any script (user) function ?

[Kurosu]

I thought some of you might be interested into seeing who is one of the guy behind SSIM, so here is a blurry, unedited picture (http://kurosu.inforezo.org/IMG_1067.JPG) of him almost live from ICIP 2005 (http://www.icip05.org/welcome.htm)

I also discussed with him a bit about this community and how he could find some info about how his algorithm is used. He'd be dearly interested into learning the actual names of the avisynth plugin authors. And maybe, if you play nice, he'll interact with all of us ;-)

Edit:Bold is correction. No time for more comments right now.

[Wilbert]

Cool! Looking at the number of papers, it must be a very big conference :) What other fun/interesting stuff did you see on that conference?

[Dams]

up little post!

[Kurosu]

Cool! Looking at the number of papers, it must be a very big conference :) What other fun/interesting stuff did you see on that conference?
Erm, this is rather off-topic. I'll bring this very discussion into private matters. And I have anyway nothing to add to the SSIM topic otherwise...

[Archimedes]

When calculating the SSIM index on still images all possible sampling windows (8x8) where selected since the sliding window moves pixel-by-pixel across the whole image.

How many sampling windows per frame are used by the SSIM filter?

Best regards
Josef

[MfA]

Exactly the same. Is the code that unreadable? :)

[mjt]

Hi all,

I would like to use SSIM 0.24 for my project but I need it in Linux environment. Has anybody implemented it in Linux or can anybody help me in making SSIM 0.24 compatiable with Linux ? Thanks a lot.

Best Regards
mjt

[MfA]

Change SSIM so it doesn't inherit from GenericVideoFilter. Change the constructor so it doesn't use PCclip and ScriptEnvironment (pass width&height explicitly). Rip out everything from SSIM::GetFrame onward in the cpp file. Make equivalent changes in the h file. Remove all the windows/avisynth specific includes and find the linux replacement for aligned_malloc/free and after that you should have something which should compile.

[thuongshoo]

Hi ! source file is wmv . I do ssim for 2nd and I have two ssim value . why ? Is that file has "2 quality" ?

[CruNcher]

thuongshoo never trust directshow it's the only tip i can give you :P

[Fizick]

Where is latest SSIM source (probably with integer lumimasking) ?

[Wilbert]

http://www.geocities.com/wilbertdijkhof/ssim-0.24a.zip

should include the fixes ...

(More good news :) I received the code of most of Clouded's plugins, and will upload it tomorrow.)

[Revgen]

http://www.geocities.com/wilbertdijkhof/ssim-0.24a.zip

should include the fixes ...

(More good news :) I received the code of most of Clouded's plugins, and will upload it tomorrow.)

Is Clouded involved in the scene anymore?

[Fizick]

Wilbert,
thanks for SSIM source..
But included binary is not corresponded to the source, it from older version.
And resource version is still 0.23.

mess,mess,mess.

[Wilbert]

Wilbert,
thanks for SSIM source..
But included binary is not corresponded to the source, it from older version.
Ok, thx! Could you fix and recompile it and upload the package in this thread?

[MfA]

You know, if you change the code from Lefungus's site you should really change the name/version number a bit ... call it .24b or something, anything.

[Fizick]

In this post tritical put a link to archive, which includes new binary.
http://forum.doom9.org/showthread.php?p=861835#post861835

I do not know where it from. Will waiting for tritical. :)
may be it is his modification? (lumimasking=0,1,2)

[Wilbert]

Well i don't know, i received the modified source from OnTech (modified by himself). Perhaps tritical also fixed it?

[MfA]

He applied the change they suggested somewhere up in this thread.

[thuongshoo]

Wilbert ! Can you upload that file to another host ? I can't access geocities.com

[MfA]

Oops I was wrong, Tritical used a binary from Sagittaire ... kinda nasty to have a binary only release of a branch, Ill priv Sagittaire and ask him to share it with source and link it in here to avoid future confusion.

Really, 9 times out of 10 (including here) it's best to just include source with modified GPLd binaries you distribute. It's not the only way to comply with the license, but it is the best way.

[tritical]

I never actually saw the source for on2tech's modified version put up anywhere, but it would be nice if someone could post it.

On another note, who wants to implement cwssim (http://www.cns.nyu.edu/~zwang/files/papers/icassp05.pdf) and wcwssim (http://www.ece.northwestern.edu/~pappas/papers/brooks_hvei06.pdf)? :D

[Fizick]

The link in Wilbert post is latest ontech's source.
http://forum.doom9.org/showthread.php?p=862104#post862104

Probably it was compiled to "Sagittaire"'s binary.

[Wilbert]

I never actually saw the source for on2tech's modified version put up anywhere, but it would be nice if someone could post it.
Ding dong: http://forum.doom9.org/showthread.php?p=862104#post862104 :)

[zambelli]

What is the correct way to plot a SSIM graph using the .csv file? I have Lumimasking=2 enabled.

Is column A the SSIM score, and column B the weight? It seems like multiplying A*B doesn't really give a realistic depiction of the encoded scenes though - the scenes which look nasty don't show up as being much lower than the scenes which look good. The graph examples posted in the first post of this thread don't really explain how one is supposed to interpret the graph.

[rbt01]

i am puzzled what is ssim.
can anybody tell me what is it?

[foxyshadis]

Start on wiki (http://en.wikipedia.org/wiki/SSIM), which has links to the home page, which links to the papers if you're interested in the technical details. You might look up CWSSIM and WCWSSIM if you're interested in variants.

[rbt01]

Start on wiki (http://en.wikipedia.org/wiki/SSIM), which has links to the home page, which links to the papers if you're interested in the technical details. You might look up CWSSIM and WCWSSIM if you're interested in variants.

I got it from your pointer
thanks a lot :)

[avdw]

Nota:
-If you use B frames under xvid, trim the first dummy frame of the xvid clip, and the last frame of the original clip
-For those who used ssim 0.1, this version is also faster !


I'm not following along, trimming the first frame of the xvid file, I understand :
b=AVISource("C:\Mymovie.avi").trim(1,0)


But how to trim the last frame of the original clip ?
I didn't find any solution in the Avisynth "Trim" explanations. :scared: