Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
|
|
Thread Tools | Search this Thread | Display Modes |
21st June 2003, 21:49 | #1 | Link |
Registered User
Join Date: Apr 2002
Location: Bretagne
Posts: 192
|
VqmCalc 0.21 & SSIM 0.1: Some attempts to compare Video Quality
Hello
I've been playing a bit with xvid lately, and the compare function of avisynth. It calculates psnr from two clips. I wasn't satisfied about psnr and searched different video quality metrics. I found JNDMetrix-IQ, a full software that use DCT, and contrast masking/temporal masking. The concept was great. http://www.sarnoff.com/products_serv...x_overview.pdf But it's patented/closed software Then i found that http://ise.stanford.edu/class/ee392j...iao_report.pdf I found it feasible, so that's what i tried to implement in the following filter VqmCalc. I'm still a newbie in c/c++. I learned coding for avisynth with this filter. So it's badly coded, slow and buggy. But i'll improve it as time pass. I can't replicate results found on the previous white-paper as some vital informations are lacking like the matrix we need to use for spatial masking. In this filter, i use MPEG Matrix with hope it reflects a bit Human Visual System. No temporal masking is implemented in this method. I've done a few experiments with a little clip encoded with xvid at constant quant2/quant4/quant6 and quant2 with VHQ1 Keep in mind these results may not reflect video quality Mean VQM: Quant2: 37.16 Quant4: 52.58 Quant6: 63.12 Quant2+VHQ1: 37.74 More points (may) means less quality. It seems to scale well with Quants but it doesn't like VHQ. It will only work with YV12. And it seems to not works with B-Frames. The clip must be mod8. If not it may crash Another bugs i've encountered are: -May crash when the log already exist -May crash during the clip, try again until it works I hope you're not afraid to test it an example: Code:
a=AVIsource("J:\Video\Source.avs") b=AVIsource("J:\Video\Quant6.avi") return VqmCalc(a,b,"results.log") [edited]: Corrected Links VqmCalc-0.21 VqmCalcSrc-0.21 SSIM-0.24 SSIMSrc-0.24 Last edited by Lefungus; 9th October 2005 at 14:52. |
21st June 2003, 22:55 | #2 | Link |
Retired AviSynth Dev ;)
Join Date: Nov 2001
Location: Dark Side of the Moon
Posts: 3,480
|
Very interesting!
The links point to the same file (binary only) - you may need to correct that - then I'll have a look at your 'nasty' source.
__________________
Regards, sh0dan // VoxPod |
22nd June 2003, 19:22 | #4 | Link |
Registered User
Join Date: Apr 2002
Location: Bretagne
Posts: 192
|
Ok, New version: 0.1
It is crashfree on my computer and maybe on others. Next step, more speed, and more tuned results. Now i use an excessively simple way to put more weight on low frequencies. I'll try to improve that Here some results from a clip, 1 Pass,H263 Matrix, constant quantizer, Koepi build: Quant 2: 40.1 Quant 3: 48.1 Quant 4: 57.9 Quant 5: 63.5 Quant 6: 70.1 Quant 4+Chroma: 57.3 Quant 4+Qpel: 59.7 Quant 4+MPEG: 56.8 Quant 4+Lumi: 58.2 Quant 4+VHQ1: 59.2 Quant 4+VHQ2: 59.2 Quant 4+VHQ3: 59.0 Quant 4+VHQ4: 58.5 |
3rd July 2003, 17:25 | #5 | Link |
Registered User
Join Date: Mar 2002
Posts: 486
|
I am very interested in this project and would like to help take it further by doing some coding myself!
First though, I want to understand the quality metric described in the paper (I'm not so happy with it as it works with 8x8 block DCTs - is it sensitive to blocking, for example?). And I want to find out if there are any other popular and open metrics in the academic press that I could implement. Anyone got any ideas? What are VQEG using? Thanks for giving me some good ideas and enthusiasm! |
3rd July 2003, 18:07 | #6 | Link |
Registered User
Join Date: Apr 2002
Location: Bretagne
Posts: 192
|
This implementation is too basic. And 8x8 DCT blocks, should effectively prevent block detection :/
I searched about interesting papers. There are some, but you'll find they don't give much informations on how to implement these algorithms. An interesting video quality metric is PDM ( Perceptual Distorsion Metric), that looks really good on paper. From Stefan Winkler. He has many papers on the web. You could search for Watson too, he created the VQM metric. There was some results from VQEG, PSNR was still quite good, but PDM was easily above. Some links http://foulard.ee.cornell.edu/public...sry_hvei03.pdf http://dewww.epfl.ch/~nadenau/Resear...view_Final.pdf http://stefan.winkler.net/Publications/chap2001.pdf |
3rd July 2003, 19:17 | #7 | Link | ||
Registered User
Join Date: Mar 2002
Posts: 486
|
Quote:
Quote:
|
||
3rd July 2003, 21:31 | #8 | Link |
Registered User
Join Date: Mar 2002
Posts: 486
|
It seems only commercial projects publish their algorithms. LeFungus found the JND algo. Here's another one http://www.its.bldrdoc.gov/n3/video/...hniques_v2.pdf
The companies also produce free evaluation software. I guess they want everyone to start using their metric. LeFungus, you said "...but it's patented/closed software". To me, this isn't the biggest problem. A source-only release under GPL is generally allowed, even where there are patents. The real problems are the complexity of making an independent implementation, and actually chosing which algo to implement. I want to know which algo will be chosen by VQEG, then I want to implenent that one |
19th July 2003, 22:49 | #9 | Link |
Registered User
Join Date: Mar 2002
Posts: 1,075
|
Ive always liked Zhou Wangs universal image quality index for its simplicity, only problem was that it broke down on uniformely colored surfaces ... he has recently published an adapted version which he calls the structure simularity index, which is still quite easy to compute.
|
19th July 2003, 23:15 | #10 | Link |
Registered User
Join Date: Apr 2002
Location: Bretagne
Posts: 192
|
http://www.cns.nyu.edu/~zwang/files/...ndex/demo.html
http://www.cns.nyu.edu/~zwang/ It's highly interesting Mfa ! And there's some code given |
21st July 2003, 11:14 | #12 | Link |
Registered User
Join Date: Mar 2002
Posts: 1,075
|
The paper for SSIM is now up at Zhou Wang's site (site is down, so for now a copy is here).
Last edited by MfA; 15th August 2003 at 18:38. |
26th July 2003, 20:16 | #13 | Link |
Registered User
Join Date: Mar 2002
Posts: 1,075
|
If anyone is interested, a decent C version of the SSIM measure from Zhou Wang is here.
Vectorization should be easy to add. Basically all for loops before "here we calculate the sum over the 8x8 block of pixels" can be vectorized, prefetching should also help for older machines (Durons). Another potential optimization is to use a circular buffer for the column sums to save on memory bandwith (on machines with 256KB+ of cache). Last edited by MfA; 15th August 2003 at 18:39. |
29th July 2003, 15:22 | #14 | Link |
Registered User
Join Date: Apr 2002
Location: Bretagne
Posts: 192
|
version updated to 0.2 for VQMCalc:
- Lots of bugs fixed. - Calculations should be correct now. But motion has no influence yet on results. version 0.1 of SSIM released: - As for VQmCalc, motion isn't taken into account - No vectorization yet, so it's quite slow. - Thanks Zhou Wang and Mfa for their code ! Results given by these plugins are experimental, and shouldn't be trusted much until some serious tests are done. |
15th August 2003, 16:54 | #17 | Link |
Registered User
Join Date: Aug 2002
Location: Hamburg
Posts: 131
|
First, thanks for these plugins. They really that, what I need to check my results.
SSIM works, but VqmCalc calculates as average "1.#QNAN0". I compare two clips made from a original DVD and a copy. So there are b-frames. But I don`t think that is related to b-frames, or ? An yes, it is slow (round about 0,5 RT with Athlon XP 1800+, 512MB, WinXP Pro). But speed doesn`t really matter. regards Arlsair
__________________
Only for Germans: <<- Big3 Anleitungen ->> <<- Alles zu DVD2SVCD ->> <<- AviSynth.org auf Deutsch ->> |
|
|