Log in

View Full Version : Custom Matrix comparison - ??? - V3


Pages : [1] 2

Soulhunter
7th September 2004, 16:39
After my participation @ the Ateme beta test, I wanna start my 3rd custom matrix comparison !!!

In context to this posts (http://forum.doom9.org/showthread.php?s=&threadid=79173&perpage=20&pagenumber=3), I wanna discuss which CQM to use for this comparison...


My suggestion:

- HVS Best Picture

- HVS Good Picture

- MPEG

- H.263

- Jawors 1CD

- EQM v2

- Soulhunters v3

- Soulhunters v6

- Didèes SixOfNine HVS


Maybe I should use some sort of knock-out system ???

- Round 1 = The 9 matrices above...

- Round 2 = The best 3 of the 1st round -vs- 6 new ones

- Round 3 = The best 3 of round 2 -vs- 6 new ones again


This way the test is not so "huge" as the one before, but is still fair... :rolleyes:

Suggestions, comments and critic plz !!!


Bye

Teegedeck
7th September 2004, 17:45
My 2 cents:

At very high bitrates SixOfNine looks better than SixOfNine-HVS.

I.e. if you aim at a transparent encoding and SixOfNine is used up to quant=4, this would look sharper than SixOfNine-HVS.

Also, you have very different contenders up there; high- and low-bitrate and universal-use matrices... What bitrate/pixel range do you want to test?

unmei
7th September 2004, 19:43
I'm glad you decided to drastically reduce the number of matrices in test. In the last one it took me one evening to only do maybe 5-10% of what were needed to do the full test seriously. Definitely too much, that's why i cowarded out :)

Also i think it is a good idea to go for a lower bitrate. It will be much easier to make out differences, and to characterise them in comments. IMO you shouldn't go higher than 1000kbit if the source will be of similar 'difficulty' as the last one.

As for which matrices, you probably in the first place want those that actually are much used/discussed on this forum. I don't think they are necessarly the best for all times, but it will make the test meaningful for the reader. After all, it should probably not be a competition of matrix authors but rather a guide to compare the matrices and characterise/categorise them.. That said, i can't give you much help on the selection as i mostly do anime and use h263 and very seldom others.

If you can't find another host, my school were probably ok with hosting short sample sequences (i think they can't look at it as pirating, and as long as the total is not too large). I could not give you FTP acess, but i had to upload them for you. Like you put it on my home FTP and i move it over to the school. PM me if once you need it.

Soulhunter
7th September 2004, 20:07
Originally posted by Teegedeck

Also, you have very different contenders up there; high- and low-bitrate and universal-use matrices... What bitrate/pixel range do you want to test?

Dunno, maybe I use again Gladiator (PAL/R2) / chapter 19 (3:57 min.) / 720x576 but this time @ 750kbps !!!


I know that most matrices of my list will suck @ this bitrate, but thats part of my concept... ;)

Picking out the three best matrices shouldn't be much difficult this way, no ???

This way the other rounds would contain the same mix of HQ/LQ matrices...

Hmm, perhaps its too complicate to lure enough ppl for this test ???


Originally posted by unmei

If you can't find another host, my school were probably ok with hosting short sample sequences (i think they can't look at it as pirating, and as long as the total is not too large). I could not give you FTP access, but i had to upload them for you. Like you put it on my home FTP and i move it over to the school. PM me if once you need it.
Thanks, I appreciate your offer... :)

But I already got a offer from a other member before !!!


Bye

Teegedeck
7th September 2004, 23:04
Originally posted by Soulhunter
Dunno, maybe I use again Gladiator (PAL/R2) / chapter 19 (3:57 min.) / 720x576 but this time @ 750kbps !!!


I know that most matrices of my list will suck @ this bitrate, but thats part of my concept... ;)
And which concept would that be? Dancing with the blocks? ;)

Sorry, but a friendly disinterest from my side.

KR
7th September 2004, 23:38
Well, I'll be happy to evaluate samples again, at 750 kbps. I actually find that putting some "bad" matrices (ie matrices that we know will fail because they are not tuned for low bitrates) in the test is a good thing. What I hope to do is not to say, at the end of the test, oh look those two matrices are the best (for this source). I'll use them, and not the others. That's it. No, I wish to understand the caracteristics of the matrices and how they affect the output. (and there is a possibility that we find a "bad" matrix better than a "good" one !)

I hope that it will again be a "blind" test.

Thanks you Soulhunter for your efforts.

Mnl
8th September 2004, 07:18
Originally posted by Soulhunter
I know that most matrices of my list will suck @ this bitrate, but thats part of my concept...

Picking out the three best matrices shouldn't be much difficult this way, no ???

This way the other rounds would contain the same mix of HQ/LQ matrices...



Excellent idea! Testing the same matrices at different bitrates would really reveal something about the matrices' different properties.

Originally posted by Soulhunter
Hmm, perhaps its too complicate to lure enough ppl for this test ???



Well I don't know - If the number of matrices weren't too large I think it should be possible to gather enough people willing to evaluate the encodes - I for one surely would volunteer :)

BTW I think the idea about using small clips instead of single frames is very good - moving pictures are so much more revealing imo.

Keep up the good work.

Didée
8th September 2004, 08:30
I think the idea about using small clips instead of single frames is very good - moving pictures are so much more revealing imo.
Seconded.

What prevented me from participating in the last test wasn't the number of candidates, but the (almost) impossibility to evaluate something reasonable out of the screenshots. I spent two-and-a-half evenings with compairing, compairing, compairing ... and in the end, I could've made up something like three or four groups, but for sure not any ranking. Compairing any two pictures from the "upper" group against the source showed that both were altered by about the same amount, just in a little "different" way. For the candidates of comparable performance, I simply saw no way to tell "A is better than B" or vice versa.

For you next test - ah, *now* it's going to be low bitrate? And I thought the last one was the low bitrate test ... :D
But Okay. If it's going to be such low bitrate, and you're going to include hi-BR matrices as well, then by all means remember to use maximal VHQ. Otherwise you could simply spare the time. Generally, XviD's motion search gets "simplified" with higher quants (i.e. shorter search range), which gives a big penalty to hi-BR matrices on low bitrate, since they produce higher quants for the same bitrate. However, syskin once told that this behaviour of the ME gets compensated by VHQ - the higher, the better.

Teegedeck
8th September 2004, 10:08
Well... For full resolution I wouldn't go lower than 900 kbps with any of these matrices except perhaps Javor's.

How about first round at 900 and 2nd round at 1700 kbps? And I mean the full movie, not only this chapter, in order to test realistic conditions and also how well the matrices scale. :devil:

Soulhunter
8th September 2004, 15:24
Originally posted by Didée
Seconded.
For you next test - ah, *now* it's going to be low bitrate? And I thought the last one was the low bitrate test ... :D

Thought this as well... ;)

But ppl requested a even lower bitrate afterwards !!!


Originally posted by Didée

But Okay. If it's going to be such low bitrate, and you're going to include hi-BR matrices as well, then by all means remember to use maximal VHQ. Otherwise you could simply spare the time. Generally, XviD's motion search gets "simplified" with higher quants (i.e. shorter search range), which gives a big penalty to hi-BR matrices on low bitrate, since they produce higher quants for the same bitrate. However, syskin once told that this behavior of the ME gets compensated by VHQ - the higher, the better.

Ok, then I gonna use VHQ4... :D


Originally posted by Teegedeck
Well... For full resolution I wouldn't go lower than 900 kbps with any of these matrices except perhaps Javor's.

*Agreement*

In the last test I used 1500kbps, but this was already the lowest "acceptable" bitrate for me...

So, as 1500kbps is close to the bitrate for a 3CD encode, 900kbps would be close to a 2CD encode !!!

Regarding realistic conditions and so... :rolleyes:


Originally posted by Teegedeck

How about first round at 900 and 2nd round at 1700 kbps?

Maybe 900kbps (2CD's), 1600kbps (3CD's) and 2200kbps (4CD's) ???

Originally posted by Teegedeck

And I mean the full movie, not only this chapter, in order to test realistic conditions and also how well the matrices scale. :devil:

*Calculating*

Ohw, more than 20x / 150min movie / 2Pass @ VHQ4... :scared:

Ok, but only if this time some more ppl participate !!!


EDIT:

Regarding -> "how well the matrices scale"

I have a problem with this...


Even with 10 (instead of 3) samples for each matrix, you wouldn't see how it acts @ the whole movie !!!

Maybe you are able to show the strength/weakness of matrix No.1 with the scenes B/D/F...

But for matrix No.2 you would need the scenes A/C/E etc. !!!


Bye

unmei
8th September 2004, 15:52
Even though is said in my last post i wouldn't go higher than 1000, it was just an idea i got from the last test. As i found it difficult to spell out differences between similar samples i thought inroducing more artifacts would introduce them in different places/circumstances depending on the matrix. However, it might well be sample sequences will aready reveal such differences without the need to screw down the bitrate. Or whatever, i will just see what i can make out of the samples you give us. Whether they are at a lower or higher bitrate :p

CruNcher
8th September 2004, 17:14
What the Hell
it's crazy to knockout HVS-Better i use it for my Extra_Detail build and it allways gave me better Detail/Picture Quality then HVS-Good or HVS-Best please bring it back in or drop one of the others instead.

Soulhunter
8th September 2004, 18:08
Originally posted by CruNcher

What the Hell
it's crazy to knockout HVS-Better i use it for my Extra_Detail build and it always gave me better Detail/Picture Quality then HVS-Good or HVS-Best please bring it back in or drop one of the others instead.
Originally posted by Soulhunter

- Round 1 = The 9 matrices above...

- Round 2 = The best 3 of the 1st round -vs- 6 new ones

- Round 3 = The best 3 of round 2 -vs- 6 new ones again

Bye

Teegedeck
8th September 2004, 18:17
Originally posted by Soulhunter
So, as 1500kbps is close to the bitrate for a 3CD encode, 900kbps would be close to a 2CD encode !!!
[...]
Maybe 900kbps (2CD's), 1600kbps (3CD's) and 2200kbps (4CD's) ???

? You use AC3 audio, I suppose??? :confused: :rolleyes:

My calculation was based upon a hypothetical 135 MB for audio and a 100-minute-movie (meaning almost 192 kbps for audio). But alright then. :)

Soulhunter
8th September 2004, 18:22
Originally posted by Teegedeck
? You use AC3 audio, I suppose??? :confused: :rolleyes:

My calculation was based upon a hypothetical 135 MB for audio and a 100-minute-movie (meaning almost 192 kbps for audio). But alright then. :)

Yes, AC3 audio n' around 150 min. duration... ;)


Bye

Mnl
8th September 2004, 20:44
Seems like I got it wrong the first time. I thought you were going to test the same matrices at different bitrates.

I still think it would be better to do it this way, as I think it would yield a lot of valuable information about how the matrices scale for different bitrates.

Testing both matrices intended for a high bitrate and matrices intended for low bitrates at the same bitrate is not fair. Even more so I do not think it will be easy to interpret the results: It would be like testing a Ferrari against a small Honda solely by driving on small crowded streets in a large city.

Would it really be so hard to find a relatively small number of often used matrices?

Of course this would require that you discarded some of the more seldom used matrices – but really I think that 10 matrices would be able to cover more than 90 percent of the matrices used.

But still - you are the boss, as it is you who are willing to spend the time doing the encodes, so you get the final word. I just wanted to voice my opinion.

EDIT:
This would make it possible to test the 10 matrices at three different bitrates, for example 750 kbps, 1500 kbps and 3000 kbps

Soulhunter
8th September 2004, 21:20
Originally posted by Mnl

Seems like I got it wrong the first time. I thought you were going to test the same matrices at different bitrates.

...

Testing both matrices intended for a high bitrate and matrices intended for low bitrates at the same bitrate is not fair. Even more so I do not think it will be easy to interpret the results: It would be like testing a Ferrari against a small Honda solely by driving on small crowded streets in a large city.

Maybe I should clarify this...

My basic idea was to use the same bitrate (now 900kbps) for all 3 knock-out rounds !!!

This way most CQM's (21) are included but the "samples per round" ratio is still low (9 samples) enough...


Finally (as Teegedeck suggested) the same procedure @ 1600kbps and 2200kbps !!!

But not immediately after the 900kbps test, perhaps with a small delay of 1 week or so... :rolleyes:


Originally posted by Mnl

Would it really be so hard to find a relatively small number of often used matrices?

Of course this would require that you discarded some of the more seldom used matrices – but really I think that 10 matrices would be able to cover more than 90 percent of the matrices used.

Well, what would be you top 10 ???

As I said before, all suggestions/comments/critic is welcome... ;)


Bye

Mnl
8th September 2004, 21:46
Originally posted by Soulhunter
Maybe I should clarify this...

My basic idea was to use the same bitrate (now 900kbps) for all 3 knock-out rounds !!!

This way most CQM's (21) are included but the "samples per round" ratio is still low (9 samples) enough...


Finally (as Teegedeck suggested) the same procedure @ 1600kbps and 2200kbps !!!


I really should get some sleep - it seems like I'm not able to get anything right at all, but now I think I understand your plan at last, and I think it is a really good one.

My only point of critique is that the number of matrices is as large as it is, I think that it may be difficult to find enough people who are sufficiently dedicated to spend the time to evaluate 21 CQM's at three different bitrates, and for the test to have any merit it is absolutely nessecery that a lot more people participate than were the case in the last comparison.

Also the large number of rounds (three per bitrate) introduce the risk of people loosing interest midway in a round, which again introduce the risk that too few people participate in one of the rounds, which could cause one of the CQM's which should have won not to be included in the final test.

My point is that the test - although it in theory is really good may be difficult to carry out because of the imo too large number of matrices.

EDIT:

My top ten:


- HVS Best Picture

- HVS Better Picture

- HVS Good Picture

- MPEG

- H.263

- Jawors 1CD

- EQM v2

- Soulhunters v3

- Didèes SixOfNine

- Didèes SixOfNine HVS

Ana
8th September 2004, 22:32
Originally posted by Mnl
My only point of critique is that the number of matrices is as large as it is, I think that it may be difficult to find enough people who are sufficiently dedicated to spend the time to evaluate 21 CQM's at three different bitrates, and for the test to have any merit it is absolutely nessecery that a lot more people participate than were the case in the last comparison.

Also the large number of rounds (three per bitrate) introduce the risk of people loosing interest midway in a round, which again introduce the risk that too few people participate in one of the rounds, which could cause one of the CQM's which should have won not to be included in the final test.
[/SIZE]

That is very good point. And the opinion of the most active voters could emphasize leading to inaccurate resault. I think that vote should have only one round and max 10 CQMs.

Perhaps we could have some poll to determinate which CQM's are most used and use that information as advice to choose which CQM's take into the test.

KR
8th September 2004, 22:41
Perhaps we could have some poll to determinate which CQM's are most used and use that information as advice to choose which CQM's take into the test.

Limiting the tests to the most popular matrices is kissing bye-bye to possible innovation and amelioration. New and unused ones have to be tested too, to find the specific areas where they might be superior to others. Of course, all these matrices have to be compared to other matrices, including the most popular ones. What I want to emphasise is to not only test the most popular.

Ana
8th September 2004, 22:49
@KR

I didn't meen that only the most popular ones should be tested, but the most popular(s) used as reference. It seems that I'm bit too tired to think things trough at this hour.

Soulhunter
8th September 2004, 23:23
Originally posted by KR
Are you interested in including them in the test (or only one, if you prefer) ?

Would need that get them... :D

If attaching doesn't work, send them via pm to me !!!


Originally posted by KR

Will it be a "blind" test ?

Yes, the same way as the previous one !!!


Originally posted by Ana

Perhaps we could have some poll to determinate which CQM's are most used and use that information as advice to choose which CQM's take into the test.

Yes, but the maximum "options" for a poll is limited to 10... :(

Would be cool if there were a way to make a poll with 30 options !!!


Originally posted by KR

Limiting the tests to the most popular matrices is kissing bye-bye to possible innovation and amelioration. New and unused ones have to be tested too, to find the specific areas where they might be superior to others. Of course, all these matrices have to be compared to other matrices, including the most popular ones. What I want to emphasis is to not only test the most popular.
Originally posted by Ana

I didn't mean that only the most popular ones should be tested, but the most popular(s) used as reference. It seems that I'm bit too tired to think things trough at this hour.

Hmm, same opinion as KR here...

How to know if we wouldn't miss a "less-known" matrix that would kick the other matrices ass ???

...

Someone knows a way to make a "30 options" poll... :confused:

Would solve this problem 1-2-3 !!!


Bye

unmei
9th September 2004, 08:14
No, but with a bit more of work you can do it by each person giving a list. Like you could collect a list to chose from, or say the list is open. A person can vote with a post only containing "i vote for" and such a post should not be edited later on (else its invalid).
This of course requires two or three things. People are ready to vote open, but this is not so much a "private" thing (i think :). Someone is willing to count these votes. The counter is trusted. The last one is hopefully not a big issue on this forum.
Oh well, this might be one kind of a stupid idea, but i tried to come up with something..

Teegedeck
9th September 2004, 09:40
At the risk of repeating myself; you could simplify it by sparing yourself to test SixOfNine (non-HVS) and other dedicated hi-bitrate matrices in the 1st round. In the 2nd and third round you don't really need to test Javor's 1CD. In the 3rd round I'd say you don't need to test h263, either.

Edit: On the other hand it would seem good to leave in h263 as a low-quality 'anchor' that can tell us how much better or worse custom matrices work compared to the default solution.

Audionut
10th September 2004, 01:57
First of all if not already mentioned/agreed apon.
All tests should be done with default xvid settings. Including, single pass const. quant. 4.

Having said that, perhaps ALL matices should be done at different quants, to see how well they scale, rather than different bitrates.
Perhaps quants. 2, 4, 6.

Those of us, who use single pass, cont. quants. could make an easy, informed decission as to which matrix is best for our needs.
Those that use 2 passes, can say, that matrix "x" is crap lower than quant 5, and thus disregard it if planing on using a low bitrate.

Also, and this will increase everything considerably. A number of different clips should be used.

Including, High Motion, Low Motion, Dark Scene, High Contrast, Anime, and any otther types of scene.

This is not a test for the faint of heart. Different Matrices, are suited to different enviroments, and should be tested in such.

My advice would be to gather good people to help with this test.
IE: one person could test high motion scenes, one low motion, etc.

Sharktooth
10th September 2004, 14:14
Uhm... my suggestion is to group the quant matrices: high bitrate,mid bitrate, low bitrate, general purpouse.
Make separated encodings respecting the matrices group bitrate then evaluate what are the best matrices for each group with a blind test.

Soulhunter
10th September 2004, 17:43
Originally posted by Teegedeck

At the risk of repeating myself; you could simplify it by sparing yourself to test SixOfNine (non-HVS) and other dedicated hi-bitrate matrices in the 1st round. In the 2nd and third round you don't really need to test Javor's 1CD. In the 3rd round I'd say you don't need to test h263, either.

Edit: On the other hand it would seem good to leave in h263 as a low-quality 'anchor' that can tell us how much better or worse custom matrices work compared to the default solution.
Originally posted by Sharktooth

Uhm... my suggestion is to group the quant matrices: high bitrate, mid bitrate, low bitrate, general purpose.
Make separated encodings respecting the matrices group bitrate then evaluate what are the best matrices for each group with a blind test.

Ok, 3 bitrates (900/1600/2200) and 8-10 matrices for each round then !!!

My suggestion for the 3 rounds...


2200kbps

- H.263

- Fox Home Entertainment

- SemiInsane

- Selur - High datarate

- Selur's Matrix

- Didée's SixOfNine

- Acaila's

- Didée's SixOfNine HVS

- Soulhunters (v3)


1600kbps

- H.263

- Didée's SixOfNine HVS

- Andreas 78er

- Soulhunters v3

- EQM v2

- HVS Best Picture

- Soulhunters v5

- Y.A.C.Q.M.

- MPEG

- HVS Better Picture


900kbps

- H.263

- HVS Better Picture

- HVS Good Picture

- Bach1

- Jawor 1CD Matrix

- AVAMAT6

- Bulletproof's Heavy Compression

- Ultimate Matrix


Not sure, give me your own suggestions plz...


Originally posted by Audionut

Having said that, perhaps ALL matrices should be done at different quants, to see how well they scale, rather than different bitrates.
Perhaps quants. 2, 4, 6.

Those of us, who use single pass, cont. quants. could make an easy, informed decision as to which matrix is best for our needs.
Those that use 2 passes, can say, that matrix "x" is crap lower than quant 5, and thus disregard it if planing on using a low bitrate.


Wish to hear other ppl's opinin about this...


Originally posted by Audionut

A number of different clips should be used.

Including, High Motion, Low Motion, Dark Scene, High Contrast, Anime, and any other types of scene.

This is not a test for the faint of heart. Different Matrices, are suited to different environments, and should be tested in such.

Sure, but it would raise the number of samples again !!!

Makes it even harder to get ppl participating... :(


Originally posted by Audionut

My advice would be to gather good people to help with this test.
IE: one person could test high motion scenes, one low motion, etc.

Ok, who wants to help... :D

As I have broad-band now, I could even mail ya the source(s) !!!


EDIT: Fixed Didée's name...


Bye

Teegedeck
10th September 2004, 17:51
Didée's SixOfNine can compete at 1600 kbps area and SixOfNine-HVS at 900 kbps (indeed I wouldn't be surprised if it ranked on top in that round). :)

Soulhunter
10th September 2004, 18:04
Originally posted by Teegedeck
Did�e's SixOfNine can compete at 1600 kbps area and SixOfNine-HVS at 900 kbps (indeed I wouldn't be surprised if it ranked on top in that round). :)

Hmm, PSNR and SSIM wise they cope much better @ higher bitrates !!!

And in this case my eyes telling me exactly the same... :D

IMO the sweet-spot should be around 2000 kbps !!!

But, let us hear some other ppl's opinion...


Bye

Teegedeck
10th September 2004, 18:12
Welllll... It is als source-dependend. I've encoded (2 CDs) Last Samurai with MPEG, SixOfNine-HVS, and Javor's 1CD and as far as I remember this was how they ranked, too. I tried Chungking Express (1 CD) with MPEG and SixOfNine-HVS and that time MPEG was clearly beaten.

Soulhunter
10th September 2004, 18:28
Originally posted by Teegedeck
Welllll... It is also source-dependent. I've encoded (2 CDs) Last Samurai with MPEG, SixOfNine-HVS, and Javor's 1CD and as far as I remember this was how they ranked, too. I tried Chungking Express (1 CD) with MPEG and SixOfNine-HVS and that time MPEG was clearly beaten.

Haven't seen Chungking Express !!!

Maybe it was much more compressible than Last Samurai...

Btw, is Chungking Express good ???


Bye

Teegedeck
10th September 2004, 18:46
It isn't Citizen Kane, but still very nice. And yes, it compresses well.

Soulhunter
10th September 2004, 19:01
Originally posted by Teegedeck

And yes, it compresses well.

Think this explains our different bitrate-aim with SixOfNine...

Good to compress @ low bitrate = Average to compress @ high bitrate !!!


Bye

Teegedeck
10th September 2004, 19:15
My valued colleague; don't forget that Chungking Express went onto 1 CD and Last Samurai on 2. They were compressed to about the same percentages of the 1st pass (peformed at quant=3), i.e. about 30%.

It is not quite as simple as you outlined it above. ;)
[sorry; I'm in a nagging mood today]

Soulhunter
10th September 2004, 20:14
Originally posted by Teegedeck

My valued colleague; don't forget that Chungking Express went onto 1 CD and Last Samurai on 2. They were compressed to about the same percentages of the 1st pass (performed at quant=3), i.e. about 30%.


- Last Samurai ~ 155 min.

- Chungking Express ~ 100 min.


But sure, this would only make a difference of 1/3 and not 1/2 !!!

Im not very good in math, sorry... ;)


Originally posted by Teegedeck

It is not quite as simple as you outlined it above. ;)

Probably we should ask Didée whats SixOfNine's supposed sweet-spot !!!


EDIT: Fixed Didée's name...


Bye

Teegedeck
10th September 2004, 20:35
Originally posted by Soulhunter
Im not very good in math, sorry... ;)
Yes, yes, rub it in! :sly:

Sharktooth
11th September 2004, 14:07
@SH: Uhm... 6of9 HVS is good for 1600 and 900 while 6of9 is better @2200
I would like to see EQM v2 results at 900, 1600 and 2200 but i think i'm asking too much.. :D
However if you can send me the source and the .avs i will do it for myself.
I wish to give some help too.

Didée
11th September 2004, 14:36
Oh my Lord.

Do with my matrices whatever you want.

But please don't spell me "Didèe", and even less "Did�e".

Soulhunter, broadband seems no good for your language setup :D

KR
11th September 2004, 15:13
Oh my Lord. (heheh)

NOW the attachment's there. Oh well, I did sent it via PM to you, Soulhunter, but it might be not important now.

It seems people willing to participate in this test wish to do it by separating the matrices according to which bitrate they were designed to be used. They will then come up with "the best matrix", for each group.Uhm... my suggestion is to group the quant matrices: high bitrate, mid bitrate, low bitrate, general purpose.
Make separated encodings respecting the matrices group bitrate then evaluate what are the best matrices for each group with a blind test.

Because of the subdivision of the test in 3 groups, it is going to take a lot of time to evaluate all samples, and including new/unused matrices in the test would make it even longer. People would likely stop participating in the test.I'm glad you decided to drastically reduce the number of matrices in test. In the last one (...) Definitely too much, that's why i cowarded out
In the last test, people complained about the "high" number of matrices. I think that the problem was the use of multiple screenshots.Didée: What prevented me from participating in the last test wasn't the number of candidates, but the (almost) impossibility to evaluate something reasonable out of the screenshots.
Using video samples is a good idea, but it will be difficult to say which matrix is better at a high bitrate like 2200 kbps.

--
OT: I wish to thank the person that gave a link to shuffle.mpg . If you find other such codec-killer scenes on the web, would you please give me some links ? Thanks.

Teegedeck
11th September 2004, 16:26
Sidenote: Well, waddayaknow... I happen to have the stats-file of Last Samurai still lurking on my PC. And Last Samurai and Chungking both seem to have about 2300 MB first-pass size at quant=3, if I haven't done anything awkwardly different back then that I don't remember. So maybe I even tried a 1-CD encode of Last Samurai back then and don't remember. Too bad I've deleted all those encodes.

Soulhunter
11th September 2004, 18:10
Originally posted by Did�e

But please don't spell me "Didèe", and even less "Did�e".

Soulhunter, broadband seems no good for your language setup :D

Uhm, watch this screenshot of your post...

http://img72.exs.cx/img72/9448/Unbenannt1.png


And now watch this one of my post...

http://img72.exs.cx/img72/4451/ghghg.png


I cant even send a pm to you !!!

No matter if I write http://img72.exs.cx/img72/4673/Unbenann2t.png or http://img72.exs.cx/img72/1734/Unbenannst.png... :(

Dunno, someone knows how to solve this ???


Tia n' Bye

Soulhunter
11th September 2004, 18:24
Originally posted by Sharktooth

I would like to see EQM v2 results at 900, 1600 and 2200 but i think I'm asking too much.. :D

For the 900 and 2200 round, there is still a free space... ;)


Originally posted by Sharktooth

I wish to give some help too.

Nice, I gonna send you a pm soon... :)


Bye

Soulhunter
11th September 2004, 18:51
Originally posted by KR

Because of the subdivision of the test in 3 groups, it is going to take a lot of time to evaluate all samples, and including new/unused matrices in the test would make it even longer...


The good thing @ this method is...

If we found "the best" matrix for each bitrate, its easier to compare new ones matrices then !!!

Think, its only -> the old "best" one -vs- the new one... ;)


Originally posted by KR

OT: I wish to thank the person that gave a link to shuffle.mpg . If you find other such codec-killer scenes on the web, would you please give me some links ? Thanks.

You mean the shuttle 1080i file from the Anteme thread ???

Yes, maybe it would be a even better source for this test... :rolleyes:

- Source is hard to encode (much smoke...)

- Source is ultra high quality (1080i -> DVD res !!!)

- Source is public (everyone is able to reproduce the results)


Bye

tiberious
11th September 2004, 19:10
Originally posted by Soulhunter

No matter if I write http://img72.exs.cx/img72/4673/Unbenann2t.png or http://img72.exs.cx/img72/1734/Unbenannst.png... :(

Dunno, someone knows how to solve this ???


Tia n' Bye [/B]

your text encoding is in Unicode (UTF-8) however Didée i guess is using
Western (ISO-8859-1). I would take a picture of what i see, but am not in a position atm, but i can say that it looks very weird.

Originally posted by Didée

But please don't spell me "Didèe" (" D i d à ¨ e "), and even less "Did�e" (" D i d ï ¿ ½ e ").


Also it is an acute accent (é) not a grave accent (è)

:]

malkion
11th September 2004, 19:25
tiberius,

just find a post by Didee, then click the PM icon right underneath it to send your PM.

Soulhunter
11th September 2004, 20:28
Hmm, changing the character encoding of FireFox (UTF/ISO) does not help... :confused:


Bye

KR
11th September 2004, 20:54
The good thing @ this method is... If we found "the best" matrix for each bitrate (for this source), its easier to compare new ones matrices then !!! Think, its only -> the old "best" one -vs- the new one...I guess I was too tired to think about that.

Yes, shuttle.mpg (not shuffle, oups) is very interesting. It's a bit dark, has smoke (easy to notice blocks), water (easy to see blurring), and luminance and color gradations (banding).

I hadn't noticed it was interlaced ! (I should get some sleep...) Resizing it to 848x480 Precise bicubic A=-0.6 kinda blended the fields... Oups.

BTW, please don't loose your time with the two matrices I sent. I got the time to test them with Video Quality Studio and their Feng Xiao VQM value is slightly worse than H.263 . Also, they need work (I read somewhere that numbers in the center should be bigger than those on lower-left and upper-right).

TheCreamCrackerBoy
12th September 2004, 02:51
Hello to all,

In my humble opinion, perhaps you should consider low bitrates (1 film = 1 700MB-cd) in this encoder comparision, since the matrices can show constrasting behaviours according to the bitrate used.

Thus, I believe I am not the only one that (still) likes to work with low bitrates (at least until I get a dvd-ram recorder). :)

At your disposal for feedback,

The Cream Cracker Boy
Manaus, Amazonas, Brazil.

CeeJay.dk
16th September 2004, 23:53
Originally posted by Soulhunter
Hmm, changing the character encoding of FireFox (UTF/ISO) does not help... :confused:


Bye

Try setting Firefox character encoding to Western (Windows-1252).
It works for me.

Soulhunter
17th September 2004, 09:42
Originally posted by CeeJay.dk

Try setting Firefox character encoding to Western (Windows-1252).
It works for me.

Ok, does it look right now... -> Didée ???


Thanks n' Bye