First Subjective test application

[everwicked]

Hello,

I just managed to get the time to pull myself together and get a first version of my subjective evaluation application out.

It is now part of Video Quality Studio (http://www.everwicked.com/vqstudio/) and you can get it from here (http://www.everwicked.com/vqstudio/).

Constructive feedback is as always appreciated.

[jggimi]

Wow, Everwicked. I've read your introductory documentation. It is extremely clear and easy to understand. I'll be trying your lab tool out on my very next encode.

[sh0dan]

Nice - a helpful addition would be SSIM (http://www.cns.nyu.edu/~zwang/files/research/ssim/index.html) measurement.

An implementation of the algorithm can be found here (http://perso.wanadoo.fr/reservoir/avisynth.html).

[everwicked]

Thanks for the comments guys.

Just to mention that the objective utility (PSNR, VQM) was released quite some time ago (see this (http://forum.doom9.org/showthread.php?s=&threadid=61107&highlight=video+quality+studio) thread).

It's the subjective part that was just released for the first time.

[Sirber]

Does not work with RMVB and AVS(RMVB) :(

[everwicked]

Originally posted by Sirber
Does not work with RMVB and AVS(RMVB) :(

Now that's a pretty vague report.

Which application of the two are we talking about, and what is the error?

[iradic]

Won't open AVS files...
Only .avi files...
In "Open" dialog only "Video files" are available as format (that is .avi is shown only in my case)...
Tried renaming avs to avi - no way - not working...

Solution?

[BoNz1]

Originally posted by iradic
Won't open AVS files...
Only .avi files...
In "Open" dialog only "Video files" are available as format (that is .avi is shown only in my case)...
Tried renaming avs to avi - no way - not working...

Solution?

Just type in the filename and extension and it will work alright, although when I was testing this earlier avs files behaved somewhat peculiarly crashing the program when you close it. everwicked knows of this though so no need to belabour the point I understand the problem is no fault of the program itself.

[everwicked]

Yeah, maybe sh0dan can shed some light to this....

Sorry about the open file dialog, I forgot to add .avs. Rest assured it's gonna be in the next version :)

In the meantime just type *.* in the dialog and it will show everything..

[DarkDudae]

Great work everwicked!

[Soulhunter]

Haven't tried this new version, but a older one...

Great tool, but one thing bugs me !!!

Could you please make it output the results also in a *.txt file ???

Or is this already possible... :confused:


Tia, Soulhunter

[everwicked]

Originally posted by Soulhunter

Could you please make it output the results also in a *.txt file ???

Or is this already possible... :confused:


Uh.... a comma seperated file (.csv) IS a text file and it's just values seperated by commands like the name suggests. Its like that because Excel can open them, but rename the file and open with whatever application you like...

[Soulhunter]

Originally posted by everwicked
Its like that because Excel can open them, but rename the file and open with whatever application you like... Mean just a result-txt-file like...

Min. PSNR = xx.xx

Avg. PSNR = xx.xx

Max. PSNR = xx.xx

And so on... ;)

Or just a info box that shows the final results...

Just because Ive no Excel !!!


Bye

[everwicked]

It's in my TODO list...

[Soulhunter]

Originally posted by everwicked
It's in my TODO list... BIG THANKS !!!

Bye

[iradic]

Thanks for help...

Another question: How to exit subjective test (it goes full screen and no way to go back, abort or something else).

Thanks again...

[everwicked]

That's on my TODO list as well..

Just as a note, you should make the clips 8-10 seconds long so that you only have to interrupt if you made a mistake or the phone rings :). Or in other words, don't try it with large files.

[iradic]

Another problem or bug (could be on my side...)?

Well, when watching subjective test video is full of artifacts (I used avs scripts - trimed 1% of movie with selectrangeevery...). When wathing these scripts in vdubmod or bsplayer everything is ok...

I'll try some more tests and see ... thanks for help

[everwicked]

I don't know about bsplayer but vdubmod uses VFW to decode avs whilst I use directshow.

So the ultimate test, would be to drag the file onto graphedit and see what happens when you play it...

[iradic]

From GraphEdit:
AviSynth video -> Avi Decompressor -> Video Renderer

But I solved problem adding pixel_type="yv12" in avifilesource.

Also for TODO list ;) after subjective test program just closes (or crashes?)...

[everwicked]

Originally posted by iradic
Also for TODO list ;) after subjective test program just closes (or crashes?)...

See above, the problem occurs using avisynth only so it's not a problem from my part..

[iradic]

so I souldn't use avs for subjective tests at all ?

[everwicked]

Uhm.. does it crash before you can view the results? I thought it crashes when you exit the application?

Let me know and I will look into it.

[iradic]

Yes, just after voting... (when I click vote) ... I will try not to vote - let the time pass and see...

[everwicked]

thats weird

i'll have a look at it

not voting will just have you stay there for ever :)

[iradic]

Don't know...
Maybe my comp setup is wrong somewhere... beacouse artifacts are here again (strange - they show only on .avs with avisource not for .avs with mpegsource)...

[everwicked]

Hello,

iradic: I tried a couple of Avisynth scripts here but didn't notice anything weird. I have 2.51 installed if that makes any difference. I will run a bit more tests though to be sure.

In other news, 0.32 is out.


Objective:

* The video interface used can be chosen while opening the reference/test files
* Other minor GUI and internal alterations

Subjective:

* DSCQS is now implemented as described in ITU-R BT.500 standard (order is randomised, two scales of 0-100, score is difference)
* The user can exit the test using ALT+F4
* An option to enable/disable resizing was added
* Other minor GUI alterations


You can get it at http://www.everwicked.com/vqstudio/

[iradic]

Dloaded new ver and it plays fine except avs are played through dshow only (trying video for win freezws program)...

this is created after pressed exit button...
**********************************************************************
Date 02/21/2004 Time 12:03
VQSTUDIO_SUBJECTIVE caused an invalid page fault in
module KERNEL32.DLL at 016f:bff8e1ad.
Registers:
EAX=c002fa54 CS=016f EIP=bff8e1ad EFLGS=00010212
EBX=0cacfe28 SS=0177 ESP=0c9cff7c EBP=0c9d0218
ECX=00000000 DS=0177 ESI=00000000 FS=2a5f
EDX=bff6682d ES=0177 EDI=bff69198 GS=0000
Bytes at CS:EIP:
53 8b 15 f4 bc fb bf 56 89 4d e4 57 89 4d dc 89
Stack dump:

**********************************************************************
Date 02/21/2004 Time 22:57
VQSTUDIO_OBJECTIVE caused an invalid page fault in
module <unknown> at ef99:059b0748.
Registers:
EAX=00000000 CS=016f EIP=059b0748 EFLGS=00010246
EBX=05ab4924 SS=0177 ESP=055afbcc EBP=055afbfc
ECX=77efc2f0 DS=0177 ESI=00000004 FS=36c7
EDX=c002fa5c ES=0177 EDI=00000004 GS=0000
Bytes at CS:EIP:

Stack dump:
057d6cbf 05ab4924 000006e0 81f4de00 7a1f47ec 00000004 00000000 00000000 055afbd8 055afe28 058ce8b4 ffffffff 00000000 77ee199a 05ab4924 00000001

[everwicked]

Can you post your .avs file too?

Also, if you can, it would be nice if you could cut a small part of the video the .avs uses and upload it somewhere.

If I can reproduce the error I might be able to do something...

[iradic]

reference avs:
--------------
# SOURCE
video=mpegsource("C:\TV-CAP\capture.mpg", 156506)
#
# CROPPING
crop(video,8,6,336,276)
#
Undot()
#
# DENOISING: little noise
Temporalsoften(2,3,3,mode=2,scenechange=6)
mergechroma(blur(1.3))
FluxSmooth(5,7)
#
# RESIZING: soft-bicubic
BicubicResize(320,240,0.333,0.333)
#
# TRIM
f_num=framecount(video)-4499
trimed=trim(last,0,f_num)
#
# SELECT RANGE = 1 %
sel_range(trimed,1)
#
function sel_range(clip c, float percent)
{
frange=floor(250./(percent/100.))
return SelectRangeEvery(c, frange, 250)
}
-------------------------------------------

test avs:
---------
# SOURCE
video=avisource("C:\TV-CAP\bulletproof-hq.avi")
#
f_num=framecount(video)-4500
trimed=trim(video,1,f_num)
#
# SELECT RANGE = 1 %
selrange(trimed,1)
#
function selrange(clip c, float percent)
{
frange=floor(250./(percent/100.))
return SelectRangeEvery(c, frange, 250)
}
-------------------------------------------

About uploading... no ... (I'll try ...)
mpg is captured mpeg-1 from pinnacle pctv
avi is xvid 1.00 rc1

[iradic]

Just to clarify qstudio crashes after I press exit -> error window -> data are then saved to faultlog...
qstudio freezes when using "video for win" -> close with task manager... no faultlog data for that...

[everwicked]

Cool, I got the files.

I will run some tests later.

Thanks.

PS: Are we talking about subjective tests only here?

[iradic]

Answer is yes - both subj and obj...
Did more tests and sometimes prog (obj and subj ver) crashes, freezes and sometimes not ... you should forget about my problems 'cause my system sucks...

Thanks anyway...

[everwicked]

Hehe there's no such thing as your system sucks :)

I will run some tests when I get the time, I am pretty busy this week but maybe sometime in the weekend.

I will post results here.

[everwicked]

VQ Subjective is much more sane now.

Try the development version from http://www.everwicked.com/vqstudio/beta/