Custom Matrix comparison - V2 (blind test)

[Soulhunter]

So, once again a CQM comparison, but this time as blind test...


1. Ok, how is it supposed to work ???

I did a encode of this source...

Gladiator (PAL/R2) / Chapter No. 19 (3:57 min.)


And this settings...

XviD v.1.0.1 (Koepis) / 2Pass @ 1500kbps (VHQ1) / I-frame interval = 250


With 27 different matrices !!!


I took 3 sample-frames of each encode, and named them like this...

CQM_NO_006_FRAME-974.jpg


The red text indicates the matrix that was used...

Which matrix belongs to which number is top-secret till the test ends !!!

CQM_NO_006_FRAME-974.jpg


The blue text indicates the sample-frame that was used...

CQM_NO_006_FRAME-974.jpg



2. But how to vote now ???

Simple, compare the CQM samples to the source sample and judge them...

IMO, the best method is to skip between them to see the significant differences !!!

When you got a good impression, write down a number between 1-10 (1= crap / 5=average / 10=superb)

Then post your votes here like this...


FRAME-974:

CQM_NO_001 = 7.0

CQM_NO_002 = 4.5

CQM_NO_003 = 6.0

And so on...


But dont post your results immediately, to prevent affecting other ppl's choice !!!

After some time (maybe 4 weeks), I will ask for the results...

If you don't wanna wait this long, you can send me your results via pm instaed !!!


Hope this explains it more or less... :rolleyes:


EDIT:

Here (http://ebola.gamersrevolt.it/CQMC/Sampleframes.zip) are all samples boundled in a zip (12 MB) !!!

But I will keep the following links for the ppl that wanna use their browsers to watch the samples... ;)

[Soulhunter]

3. So, here are the samples...


Frame No. 974

- Souce (http://img16.exs.cx/my.php?loc=img16&image=Source_FRAME-974.jpg)

- CQM No. 001 (http://img16.exs.cx/my.php?loc=img16&image=CQM_NO_001_FRAME-974.jpg)

- CQM No. 002 (http://img13.exs.cx/my.php?loc=img13&image=CQM_NO_002_FRAME-974.jpg)

- CQM No. 003 (http://img10.exs.cx/my.php?loc=img10&image=CQM_NO_003_FRAME-974.jpg)

- CQM No. 004 (http://img10.exs.cx/my.php?loc=img10&image=CQM_NO_004_FRAME-974.jpg)

- CQM No. 005 (http://img16.exs.cx/my.php?loc=img16&image=CQM_NO_005_FRAME-974.jpg)

- CQM No. 006 (http://img16.exs.cx/my.php?loc=img16&image=CQM_NO_006_FRAME-974.jpg)

- CQM No. 007 (http://img16.exs.cx/my.php?loc=img16&image=CQM_NO_007_FRAME-974.jpg)

- CQM No. 008 (http://img16.exs.cx/my.php?loc=img16&image=CQM_NO_008_FRAME-974.jpg)

- CQM No. 009 (http://img14.exs.cx/my.php?loc=img14&image=CQM_NO_009_FRAME-974.jpg)

- CQM No. 010 (http://img14.exs.cx/my.php?loc=img14&image=CQM_NO_010_FRAME-974.jpg)

- CQM No. 011 (http://img14.exs.cx/my.php?loc=img14&image=CQM_NO_011_FRAME-974.jpg)

- CQM No. 012 (http://img3.exs.cx/my.php?loc=img3&image=CQM_NO_012_FRAME-974.jpg)

- CQM No. 013 (http://img10.exs.cx/my.php?loc=img10&image=CQM_NO_013_FRAME-974.jpg)

- CQM No. 014 (http://img10.exs.cx/my.php?loc=img10&image=CQM_NO_014_FRAME-974.jpg)

- CQM No. 015 (http://img6.exs.cx/my.php?loc=img6&image=CQM_NO_015_FRAME-974.jpg)

- CQM No. 016 (http://img6.exs.cx/my.php?loc=img6&image=CQM_NO_016_FRAME-974.jpg)

- CQM No. 017 (http://img28.exs.cx/my.php?loc=img28&image=CQM_NO_017_FRAME-974.jpg)

- CQM No. 018 (http://img28.exs.cx/my.php?loc=img28&image=CQM_NO_018_FRAME-974.jpg)

- CQM No. 019 (http://img28.exs.cx/my.php?loc=img28&image=CQM_NO_019_FRAME-974.jpg)

- CQM No. 020 (http://img28.exs.cx/my.php?loc=img28&image=CQM_NO_020_FRAME-974.jpg)

- CQM No. 021 (http://img13.exs.cx/my.php?loc=img13&image=CQM_NO_021_FRAME-974.jpg)

- CQM No. 022 (http://img13.exs.cx/my.php?loc=img13&image=CQM_NO_022_FRAME-974.jpg)

- CQM No. 023 (http://img13.exs.cx/my.php?loc=img13&image=CQM_NO_023_FRAME-974.jpg)

- CQM No. 024 (http://img32.exs.cx/my.php?loc=img32&image=CQM_NO_024_FRAME-974.jpg)

- CQM No. 025 (http://img32.exs.cx/my.php?loc=img32&image=CQM_NO_025_FRAME-974.jpg)

- CQM No. 026 (http://img32.exs.cx/my.php?loc=img32&image=CQM_NO_026_FRAME-974.jpg)

- CQM No. 027 (http://img32.exs.cx/my.php?loc=img32&image=CQM_NO_027_FRAME-974.jpg)

[Soulhunter]

Frame No. 2250

- Source (http://img10.exs.cx/my.php?loc=img10&image=Source_FRAME-2250.jpg)

- CQM No. 001 (http://img10.exs.cx/my.php?loc=img10&image=CQM_NO_001_FRAME-2250.jpg)

- CQM No. 002 (http://img11.exs.cx/my.php?loc=img11&image=CQM_NO_002_FRAME-2250.jpg)

- CQM No. 003 (http://img11.exs.cx/my.php?loc=img11&image=CQM_NO_003_FRAME-2250.jpg)

- CQM No. 004 (http://img11.exs.cx/my.php?loc=img11&image=CQM_NO_004_FRAME-2250.jpg)

- CQM No. 005 (http://img36.exs.cx/my.php?loc=img36&image=CQM_NO_005_FRAME-2250.jpg)

- CQM No. 006 (http://img36.exs.cx/my.php?loc=img36&image=CQM_NO_006_FRAME-2250.jpg)

- CQM No. 007 (http://img4.exs.cx/my.php?loc=img4&image=CQM_NO_007_FRAME-2250.jpg)

- CQM No. 008 (http://img4.exs.cx/my.php?loc=img4&image=CQM_NO_008_FRAME-2250.jpg)

- CQM No. 009 (http://img4.exs.cx/my.php?loc=img4&image=CQM_NO_009_FRAME-2250.jpg)

- CQM No. 010 (http://img33.exs.cx/my.php?loc=img33&image=CQM_NO_010_FRAME-2250.jpg)

- CQM No. 011 (http://img33.exs.cx/my.php?loc=img33&image=CQM_NO_011_FRAME-2250.jpg)

- CQM No. 012 (http://img33.exs.cx/my.php?loc=img33&image=CQM_NO_012_FRAME-2250.jpg)

- CQM No. 013 (http://img14.exs.cx/my.php?loc=img14&image=CQM_NO_013_FRAME-2250.jpg)

- CQM No. 014 (http://img14.exs.cx/my.php?loc=img14&image=CQM_NO_014_FRAME-2250.jpg)

- CQM No. 015 (http://img15.exs.cx/my.php?loc=img15&image=CQM_NO_015_FRAME-2250.jpg)

- CQM No. 016 (http://img15.exs.cx/my.php?loc=img15&image=CQM_NO_016_FRAME-2250.jpg)

- CQM No. 017 (http://img15.exs.cx/my.php?loc=img15&image=CQM_NO_017_FRAME-2250.jpg)

- CQM No. 018 (http://img28.exs.cx/my.php?loc=img28&image=CQM_NO_018_FRAME-2250.jpg)

- CQM No. 019 (http://img30.exs.cx/my.php?loc=img30&image=CQM_NO_019_FRAME-2250.jpg)

- CQM No. 020 (http://img2.exs.cx/my.php?loc=img2&image=CQM_NO_020_FRAME-2250.jpg)

- CQM No. 021 (http://img2.exs.cx/my.php?loc=img2&image=CQM_NO_021_FRAME-2250.jpg)

- CQM No. 022 (http://img2.exs.cx/my.php?loc=img2&image=CQM_NO_022_FRAME-2250.jpg)

- CQM No. 023 (http://img2.exs.cx/my.php?loc=img2&image=CQM_NO_023_FRAME-2250.jpg)

- CQM No. 024 (http://img38.exs.cx/my.php?loc=img38&image=CQM_NO_024_FRAME-2250.jpg)

- CQM No. 025 (http://img14.exs.cx/my.php?loc=img14&image=CQM_NO_025_FRAME-2250.jpg)

- CQM No. 026 (http://img14.exs.cx/my.php?loc=img14&image=CQM_NO_026_FRAME-2250.jpg)

- CQM No. 027 (http://img14.exs.cx/my.php?loc=img14&image=CQM_NO_027_FRAME-2250.jpg)

[Soulhunter]

Frame No. 4997

- Source (http://img14.exs.cx/my.php?loc=img14&image=Source_FRAME-4997.jpg)

- CQM No. 001 (http://img30.exs.cx/my.php?loc=img30&image=CQM_NO_001_FRAME-4997.jpg)

- CQM No. 002 (http://img30.exs.cx/my.php?loc=img30&image=CQM_NO_002_FRAME-4997.jpg)

- CQM No. 003 (http://img19.exs.cx/my.php?loc=img19&image=CQM_NO_003_FRAME-4997.jpg)

- CQM No. 004 (http://img19.exs.cx/my.php?loc=img19&image=CQM_NO_004_FRAME-4997.jpg)

- CQM No. 005 (http://img19.exs.cx/my.php?loc=img19&image=CQM_NO_005_FRAME-4997.jpg)

- CQM No. 006 (http://img19.exs.cx/my.php?loc=img19&image=CQM_NO_006_FRAME-4997.jpg)

- CQM No. 007 (http://img29.exs.cx/my.php?loc=img29&image=CQM_NO_007_FRAME-4997.jpg)

- CQM No. 008 (http://img29.exs.cx/my.php?loc=img29&image=CQM_NO_008_FRAME-4997.jpg)

- CQM No. 009 (http://img29.exs.cx/my.php?loc=img29&image=CQM_NO_009_FRAME-4997.jpg)

- CQM No. 010 (http://img29.exs.cx/my.php?loc=img29&image=CQM_NO_010_FRAME-4997.jpg)

- CQM No. 011 (http://img40.exs.cx/my.php?loc=img40&image=CQM_NO_011_FRAME-4997.jpg)

- CQM No. 012 (http://img40.exs.cx/my.php?loc=img40&image=CQM_NO_012_FRAME-4997.jpg)

- CQM No. 013 (http://img40.exs.cx/my.php?loc=img40&image=CQM_NO_013_FRAME-4997.jpg)

- CQM No. 014 (http://img40.exs.cx/my.php?loc=img40&image=CQM_NO_014_FRAME-4997.jpg)

- CQM No. 015 (http://img40.exs.cx/my.php?loc=img40&image=CQM_NO_015_FRAME-4997.jpg)

- CQM No. 016 (http://img11.exs.cx/my.php?loc=img11&image=CQM_NO_016_FRAME-4997.jpg)

- CQM No. 017 (http://img11.exs.cx/my.php?loc=img11&image=CQM_NO_017_FRAME-4997.jpg)

- CQM No. 018 (http://img19.exs.cx/my.php?loc=img19&image=CQM_NO_018_FRAME-4997.jpg)

- CQM No. 019 (http://img19.exs.cx/my.php?loc=img19&image=CQM_NO_019_FRAME-4997.jpg)

- CQM No. 020 (http://img19.exs.cx/my.php?loc=img19&image=CQM_NO_020_FRAME-4997.jpg)

- CQM No. 021 (http://img19.exs.cx/my.php?loc=img19&image=CQM_NO_021_FRAME-4997.jpg)

- CQM No. 022 (http://img19.exs.cx/my.php?loc=img19&image=CQM_NO_022_FRAME-4997.jpg)

- CQM No. 023 (http://img19.exs.cx/my.php?loc=img19&image=CQM_NO_023_FRAME-4997.jpg)

- CQM No. 024 (http://img14.exs.cx/my.php?loc=img14&image=CQM_NO_024_FRAME-4997.jpg)

- CQM No. 025 (http://img14.exs.cx/my.php?loc=img14&image=CQM_NO_025_FRAME-4997.jpg)

- CQM No. 026 (http://img14.exs.cx/my.php?loc=img14&image=CQM_NO_026_FRAME-4997.jpg)

- CQM No. 027 (http://img14.exs.cx/my.php?loc=img14&image=CQM_NO_027_FRAME-4997.jpg)


Bye

[Manao]

Interesting.

If you could provide a zip containing all the pictures, it would be much more simple to watch the pictures.

The first source frame ( 974 ) also shows a lot of ringing / blocking, I would not have taken it in the comparison.

I'll vote when I'll have clicked on all the links you provided :)

Edit :

Some new questions :
* what was the quantizer of the different frames, or at least of the H263 one ?
* do we have to judge in comparison to the source ( ie the closer to the source the better ), or to the visual impression when the frame is considered alone ?
* are there b-frames / i-frames, and are all the frames the same type ?

[Soulhunter]

Originally posted by Manao
If you could provide a zip containing all the pictures, it would be much more simple to watch the pictures.
If I would know a (free) place that allows ZIP files... :rolleyes:


Originally posted by Manao
The first source frame ( 974 ) also shows a lot of ringing / blocking, I would not have taken it in the comparison.


Yes, I know...

IMO its good to have such a frame in the comparison as well !!!

Thought, its interesting to compare the CQM's to a "already bad" input... ;)


Originally posted by Manao
* what was the quantizer of the different frames, or at least of the H263 one ?If you wanna know, it was Q8 I-frame !!!


Originally posted by Manao
* do we have to judge in comparison to the source ( ie the closer to the source the better ), or to the visual impression when the frame is considered alone ?As I wrote, you should compare its quality in context to the source... ;)


Originally posted by Manao
* are there b-frames / i-frames, and are all the frames the same type ? No, they are different !!!

I considered to mark them with a "farmetype" info...

But while writing them down, I realized a connection between the frame-type n' matrix-type that was used !!!

When the farmetype was differently, there where always two groups...

- Low compression CQM's = I-farme

- High compression CQM's = P-farme

So, if I would post the frametype-info, this wouldn't be a real blind-test anymore... ;)


Bye

[Manao]

Low compression = I-frame

High compression = P-frameIt's the contrary :)

But if on the same frame you can have some samples with p-frames, others with b-frames, it's not a fair comparison, and it's useless, IMO.If you wanna know, it was Q8 I-frame !!!And for the frames 2250 and 4997 ?

[Soulhunter]

Originally posted by Manao
It's the contrary :)
Ehm, what do you mean... :confused:

Note: I wrote low-compression not low compressing !!!

[EDIT:]

Gah, wait... That makes no sense !!!

It should be...

Note: I wrote low-compression not low quality !!!


Originally posted by Manao
But if on the same frame you can have some samples with p-frames, others with b-frames...I haven't used b-frames !!!


Originally posted by Manao
And for the frames 2250 and 4997 ?
2250 = Q9 P-frame / 4997 = Q4 P-frame


Bye

[Manao]

"Low compression CQM" means "it doesn't compress a lot", doesn't it ?
2250 = Q9 P-frame / 4997 = Q4 P-frameThank you, so low quality matrices may saturate on the last frame. BTW, did you capped the quantizers to 2-31 or 1-31 ? ( I guess 2-31 )
I havent used b-frames !!!Doing so, it's easier to have the same frame type in order to compare the quality fairly, but it makes the test biased, because CQMs don't all behave in the same way with b-frames, and we almost all use b-frames in our daily encodes.

[Soulhunter]

Originally posted by Manao
"Low compression CQM" means "it doesn't compress a lot", doesn't it ?Or it does a low compression... :D

- High (high level of...) = strong/powerfully = more compression (lower filesize)

- Low (low level of...) = weak/light = less compression (higher filesize)


Interesting to see how two ppl can have a complete different concept for the meaning of a word !!!

No matter, I know what you mean, and you know what I mean... :)


Originally posted by Manao
Thank you, so low quality matrices may saturate on the last frame.
Nope...

Even the "lowest" quality matrix has a minimum quant of 3 !!!


Originally posted by Manao
BTW, did you capped the quantizers to 2-31 or 1-31 ?
Yes, capped them to 2-31 !!!

Think it would be better to note this in the posted settings... :rolleyes:


Originally posted by Manao
...because CQMs don't all behave in the same way with b-frames, and we almost all use b-frames in our daily encodes.
Yes, normally I use b-frames too...

But I don't use them in my tests coz it makes comparing even more difficult !!!

Like: Encode A uses a P-frame where encode B used a b-frame etc...

Limiting the chance to I&P frames only, makes it much easier IMO !!!

But yes, it limits the ability of the encoding as well... :rolleyes:


For future comparisons...

The different frametype problem:

Do you think I should force keyframes via zones next time ???

I faer it would destroy the natural frame distribution...

So it could give misleading results n' even more discussion !!!


Bye

[Manao]

Interesting to see how two ppl can have a complete different concept for the meaning of a word !!!No, we have the same definition ( "it doesn't compress a lot" and "high filesize" ). But I don't see how low compression / high quality matrices could end up having I-frames where high compression / low quality matrices put a P-frame

[Soulhunter]

Originally posted by Manao
I don't see how low compression / high quality matrices could end up having I-frames where high compression / low quality matrices put a P-frame

Should it be the other way around ???

I don't know, its something I've accidentally noticed while comparing the frametypes of the samples... :p

Is there a way (frame type/size distribution analyzer) to find out why this happened ???


Originally posted by Manao
No, we have the same definition ( "it doesn't compress a lot" and "high filesize" )

Gah, sometimes I simply read way to fast... :o


Sorry n' Bye

[Sharktooth]

Originally posted by Soulhunter
If I would know a (free) place that allows ZIP files... :rolleyes:SH check your PM :)

[Soulhunter]

Originally posted by Sharktooth
SH check your PM :)

Thanks again Sharktooth...

Ok, a zip with all samples is now access able here (http://ebola.gamersrevolt.it/CQMC/Sampleframes.zip) !!!

And yes, I will add this link to the first post as well... ;)


Bye

[lexor]

I'm not sure if this is part of the big secret, but is sample of the same number, say 24, always the same CQM? Because quality jumps between frames 4997 and 2250 is astonishing for the same number. Except for Matrices that suck consistantly on all frames :D

[Soulhunter]

Sure this info would not affect your assessment ???

No further comments... :p


Bye

[Gambler's Ruin]

Man this is an eye-killer activity.

[Soulhunter]

Originally posted by Gambler's Ruin
Man this is an eye-killer activity.
Yeah, now you know why its called a blind test... :D


Bye

[unmei]

ROFL so true

the sheer number of matrices in test makes this difficult. I spent like 90 minutes now only getting a first impression or the first 15 CQM on the first sample :O

But it's a good thing you set this test up, thank you Soulhunter. I never used QCMs because i never knew where to begin, but this test will probably give a good starting point :)

[Alxemi]

Congratulations for such a good test!!
It´s true that´s it´s difficult because of the matrices number... but the results will be very interesting :)

BTW I´ve been looking at them and they seem to have poor quality at 1500kbit... but even the original doesn´t look so great.

Thanks for your effort Soulhunter.

[Soulhunter]

Originally posted by Alxemi
BTW I´ve been looking at them and they seem to have poor quality at 1500kbit... but even the original doesn´t look so great.Mainly course the source is 90% high motion... :D


Bye

[Soulhunter]

I think its time to finish this test...

So, please post you results (here or via pm) till next Friday (the 6th August) !!!


Bye

[len0x]

I wonder how many results did you get so far ?

[Soulhunter]

So far only 3 people mailed me their results... :rolleyes:


Bye

[len0x]

Bummer, but I think it could be expected due to the number of matrixes to compare. I think first stage should be select 10 better matrixes without places, then 5 and only then ask ppl to put them in order.

[Soulhunter]

True, but then everyone would scream...

- Where is my beloved XYZ matrix ???

- With matrix XYZ the result would be much better !!!


Btw, I think it has also to do with holiday-factor...

IMO the overall activity has gone down the last weeks, not only @ this forum !!!

Hey, vacations, good weather, who wants to spend hours inside to compare CQM's... :D


Bye

[len0x]

You don't say which matrixes get selected to the next round, just publish numbers again - that way no one screams :)

[Soulhunter]

Meant afterwards...

Maybe I should delay the end another month ???


Bye

[len0x]

we could try asking doom9 to get some publicity of the subject in the news and see what's gonna happen...

[Soulhunter]

You think its "worthy enough" to get mentioned @ the main page... :o


Bye

[len0x]

may be not, but certainly can attract some ppl to it :)

[Soulhunter]

Think you are right... :)


Bye

[Ana]

I think that delaying the test to the end of month is a good idea. Atleast in europe august is the holiday month (And it would suit well to me :D ). A mention in the news page, could do some good. But there is sooo much else to do in summer... Perhaps I should forget my fishing trip tomorrow and finally take a look at the pictures.

[Blue_MiSfit]

Im at work right now, but will utalize their DSL to download the images and do a comparison when I get home.

Dissapointing that only 3 people contributed so far. Dammmit I will make a difference just like my guidance councilor said hehe ;)

~misfit

[jon.schaffer]

A word from a lazy guy:

I found the test interesting and spent some time on it, making a rough rating, but giving a note to each CQM is much more work: I did not get the courage to classify the CQM of poor quality, and prefered to focus on the better ones...

So, I would second the len0x's remark:
...first stage should be select 10 better matrixes without places, then 5...

Would a top 10 or 5 be sufficient ?
Behind this is another question: absolute notes are hard to give, would a relative rating (e.g. a>b>c... with ">" = "better than") be good?

Thanks for your effort,
Jon

PS:
there are 2 frames being exactly the same (same "content" - same file size, too) in the frame_974 test, and not in the others... could it be an error in the zipped package or can this result of the properties of two matrices: being different on some frames and exactly the same on some others ?

[Soulhunter]

Originally posted by jon.schaffer
Would a top 10 or 5 be sufficient ?
Behind this is another question: absolute notes are hard to give, would a relative rating (e.g. a>b>c... with ">" = "better than") be good?Ok then, post me your top 5 instead !!!!

Originally posted by jon.schaffer
There are 2 frames being exactly the same...
Its no error... ;)

Improbable but possible, two matrices produced exactly the same frame !!!


Bye

[Dibek]

I've done a lot of reading in this post and other post regarding matrices. The test is exelent and so far i like number 006 the most.
This is purely from vision perspective point of view.

[Soulhunter]

Ok, another month is over...

Only 7 people (includes me) have voted so far !!!


Anyhow, I gonna post the "strange" results now (only to give you something to discuss)... :D


1. Numbers and the appropriate matrices

CQM_NO_001 = EQM v2

CQM_NO_002 = Bulletproof's Heavy Compression

CQM_NO_003 = Semi-Insane

CQM_NO_004 = Jawors 1CD

CQM_NO_005 = Y.A.C.Q.M

CQM_NO_006 = Flat 8/16

CQM_NO_007 = Soulhunters v5

CQM_NO_008 = Didèes SixOfNine Max24

CQM_NO_009 = EQM v1

CQM_NO_010 = Bulletproof's High Quality Matrix

CQM_NO_011 = Soulhunters v6

CQM_NO_012 = HVS Better Picture

CQM_NO_013 = H.263

CQM_NO_014 = Andreas 78er

CQM_NO_015 = AVAMAT6

CQM_NO_016 = Acailas

CQM_NO_017 = Soulhunters v3

CQM_NO_018 = Selurs high datarate

CQM_NO_019 = HVS Good Picture

CQM_NO_020 = Bach1

CQM_NO_021 = MPEG

CQM_NO_022 = Angel BestLow

CQM_NO_023 = Fox Home Entertainment

CQM_NO_024 = HVS Best Picture

CQM_NO_025 = Selurs Matrix

CQM_NO_026 = 6of9 -vs- Jawors

CQM_NO_027 = mp4 guy's



2. Average voting results

8 points:

- Soulhunters v3

- HVS Good Picture

7 points:

- Y.A.C.Q.M

- Flat 8/16

- Soulhunters v6

- HVS Better Picture

- H.263

- Andreas 78er

- Acailas

- Selurs high datarate

- MPEG

6 points:

- HVS Best Picture

- Semi-Insane

- Jawors 1CD

- Didèes SixOfNine Max24

- Bulletproof's High Quality Matrix

- Angel BestLow

- Fox Home Entertainment

- Selurs Matrix

- 6of9 -vs- Jawors

5 points:

- EQM v2

- Bulletproof's Heavy Compression

- Soulhunters v5

- EQM v1

- AVAMAT6

- Bach1

4 points:

- mp4 guy's


One of my matrices @ the 1st place (together with HVS Good Picture) ???

Believe me, I haven't manipulated the results... :p


Bye

[Soulhunter]

Shocking, scary, unexpected... :eek:


- The Flat 8/16 matrix got a very high rating

- The matrices H.263 and MPEG got the same rating

- EQM v1/v2 & my Soulhunters v5 got the same rating as AVAMAT6 & Bach1

- Didèes SixOfNine HVS (old version) got a lower rating than the standard MPEG matrix


Bye

[KR]

I'm surprised... HVS Good having 8 points ? I did vote for Soulhunters v3 in 1st place, but HVS Good was 15th for me (worse than H.263, which I put 3rd !).

Also, I though the 2 worst ones were Semi-Insane and Fox Home Entertainment. But they both got 6 points !? Well, I only did the test for frame #4997, so maybe those matrices performed well on other frames.

Now, WHERE IS MY BELOVED XYZ MATRIX ? heheh

Thanks Soulhunters for this test and your matrix

I'm curious: Let's redo this test at 800 kbit/s

[CruNcher]

jep im also for a low bitrate one :)

[Soulhunter]

Too early for the next round, anyhow lets start discussion about it...

- Should I do it the same way, or change something ???

- Which source should I use (the same one...) ???

- How much samples per matrix (1,3,5...) ???

- Which bitrate (maybe 750kbps...) ???

- How much matrices (10-15...) ???

- Which matrices (dunno...) ???


Tia n' Bye

[Teegedeck]

Make a preselection. The sheer mass of samples made me obstain from taking part (I never could have seriously tested that many samples). Discuss how many and which matrices to test (new thread), run a poll if you still have too many candidates after the discussion. I'd say no more than 8 Matrices per round.

Have separated tests for high and low quantizers, maybe even even sequences for download. For that better select a 3-second sequence where you can actually tell differences without going frame-per-frame because that would contradict the whole idea (i.e. don't use an action-sequence)

:)

[Soulhunter]

Originally posted by Teegedeck

Make a preselection. The sheer mass of samples made me abstain from taking part (I never could have seriously tested that many samples). Discuss how many and which matrices to test (new thread), run a poll if you still have too many candidates after the discussion. I'd say no more than 8 Matrices per round.

Ok, I will open a new thread this evening... :)


Originally posted by Teegedeck

Have separated tests for high and low quantizers, maybe even even sequences for download. For that better select a 3-second sequence where you can actually tell differences without going frame-per-frame...

Uploading 3 sec. samples would be great, yes !!!

But I dunno where to get some free websapce... :(


Originally posted by Teegedeck

...because that would contradict the whole idea (i.e. don't use an action-sequence)


Dont you think that fast motion scenes are indispensable to make a fair comparison... :confused:

High compression matrices should theoretical show their strength there, no ???


Bye

[Teegedeck]

Originally posted by Soulhunter
High compression matrices should theoretical show their strength there, no ???Bye
At low bitrates, yes. You certainly are right there.
But generally you can get away with more blocks in a high-bitrate scene as long as you don't look at the individual frames.

[Sharktooth]

Ok, after accurately looking at all the frames i couldnt judge which one was the best matrix.
But i clearly distinguished low bitrate matrices from the general purpouse and high bitrate ones.
After SH unvelied what were the matrices pic numbers, i did my comparison again and i found i was right.
My impression is that low bitrate matrices performed really well on the Sample_01 frames (high motion). The reason is an extreme high motion scene requires a lot of bits and the rate control is highly stressed. A low bitrate matrix "averages" that huge bit requirements allowing the codec rate control to perform better.
See how h.263/jawor's (and other low bitrate) matrices perform in Sample_01.
Then compare those matrices in Sample_02 (mid motion) and Sample_03 (low motion).
You can clearly see how high-bitrate matrices performs better in Sample_02 and extremely better in Sample_03.
6of9HVS is performing good in every situation due to its coefficients distribuition and HVS-Better too, providing a good amount of details where they counts.
Those are "MY" reccomended matrices for mixed high-low motion movies (or "general purpouse" if you prefer), but dont expect the maximum details from them in every movie scene or in a all high or all low motion movie.

[JarrettH]

Where can I download Flat 8/16 and Y.A.C.Q.M?:confused:

I liked Soulhunter v6 over v3, Y.A.C.Q.M looks good too but I can't find it. I can't tell if Flat 8/16 has more detail then Y.A.C.Q.M, but it has less smoothing.

[JarrettH]

I'm going to do Lost in Translation with Soulhunter v6 and v3, Y.A.C.Q.M, and Flat 8/16. 1450kbps with AC3 audio will fit 2CDs.

[Soulhunter]

Y.A.C.Q.M. was my newest development... :D

Y.A.C.Q.M. - YET ANOTHER CUSTOM QUANT MATRIX

http://img73.exs.cx/img73/1747/YACQM1.png


Looks strange, but seems to work well !!!

And "Flat 8/16" is simply -> Intra matrix = All 8 / Inter matrix = All 16... ;)


Bye

[JarrettH]

Thanks, I made it in the little editor.:p