About matrix [Archive] - Doom9's Forum

feedback

30th December 2005, 05:21

O.K. I made a slight mistake.

In the Fox Home Entertainment Matrix and the High/High Matrix there is a difference.

Counting left to right as 0,1,2,3,4,5 etc. and down 0,1,2,3,4,5, etc., we look at the 3rd row 4th column the value is 13 in Fox but in 'High/High' Matrix the value is 14.

I don't know if this was a transposing error or not... Manono might know as he helped Rockas with the Matrices Compilation and the descriptions.

Personally, I use the Fox Entertainment Matrix instead of High/High when there is a need for a high bitrate matrix. But, I was using Fox before the existence of the High/High matrix in RME.

BTW, I don't if you are aware of it but you can Extract Matrices from VOB's with the Rebuilder Matrix Editor which one can then use in other movies....a very nice feature.

Regards,:)

manono

30th December 2005, 12:12

Nice find feedback-

The Fox Home Entertainment Matrix wasn't used only on Fox DVDs, but also on Warner Home Video DVDs and on Criterion Collection DVDs (and maybe on others). Fox no longer uses it, by the way, at least not the last dozen or so classic film DVDs from them that I've checked. They now use an even better matrix that compresses about 6% worse in my tests than does the Fox Matrix, making it about 6% sharper, retaining even more detail. I've just checked an older Fox DVD, a WHV DVD and a Criterion DVD. Fox and WHV use the version with the 13. Criterion, however, uses that slight variation that you discovered with a 14. So, not noticing the difference, I had grabbed the matrix off of a Criterion DVD for the Rockas Matrix Editor. In addition, I just checked the version included with LigH's Custom Quantization Matrix Editor, and his version of the Fox Home Entertainment Matrix also has that 14 instead of a 13. Maybe there are different versions of it floating around. I don't guess it makes much difference in the output quality. I did cover myself in the notes for High_High by saying:
This or something very similar is used by the best studios for their best DVD releases...
Here's what Fox DVDs use these days, in case you're interested:

08 08 09 09 10 10 11 11
08 09 09 10 10 11 11 12
09 09 10 10 11 11 12 12
09 10 10 11 11 12 13 13
10 10 11 11 12 13 13 14
10 11 11 12 13 13 14 15
11 11 12 13 13 14 15 15
11 12 12 13 14 15 15 16

08 08 09 09 10 10 11 11
08 09 09 10 10 11 11 12
09 09 10 10 11 11 12 12
09 10 10 11 11 12 13 13
10 10 11 11 12 13 13 14
10 11 11 12 13 13 14 15
11 11 12 13 13 14 15 15
11 12 12 13 14 15 15 16

I mostly buy classic film DVDs, and don't know for sure what Fox uses for new movie DVDs. It's probably the same though, as it's a great matrix. It's so good, that I don't often get a chance to use it myself.

feedback

31st December 2005, 02:59

@manono

I thought I had seen a slight difference in the two matrices previously but I wanted to make sure, as I was giving advice to Gibuk and accurate advice is the best advice, right? :D
(yes I know I used best) :p

BTW, thanks for the New Matrix. Would you suggest using it at say around the 4800+ bitrate mark? (I know I'll try it on something and see for myself ;) )

I like trying different Matrices and I am so amazed at the crappy ones that some studios use. I mean, looking at the cost involved in making a movie and then the studios go and cobble it together using some less than desireable Matrix.

On another note, I also found the "Fox Home Entertainment" matrix (with the 13) on Cinderella Man R1 W/S version. It is a Universal Pictures, Miramax Films, Imagine Entertainment movie.

I don't know if I can use the latest Fox Matrix (what do you call it? Fox 2 Matrix?!?!) on Cinderella Man as it is only about 3900 bitrate, movie only.
I am actually pushing it, I think, by using the Fox 1 or original Fox Home Entertainment matrix. (We have got to get a handle on these names... I am starting to confuse myself) :D

As the movie Cinderella Man has the Fox H.E. matrix I would think that is the way it was originally intended to be...ie. with the 13 value. So, that said I suppose the Fox Matrix with the 14 should probably be called the Criterion Matrix as that is where you found it or the Modified Fox Matrix or maybe the Manono Matrix ;) which does have a ring to it.

BTW, what do you want to call this new matrix? Really, for consistency's sake I think it needs a name and one we will stick to. I mean if someone requests a high detail matrix and I give them your new matrix and you also give it to them it needs a common name don't you think?
I was referring to it as Fox 2 as it was the second in the series of the Fox matrices (even though they don't use FHE matrix anymore per your info.) but whatever name you think is good or discriptive, I will rename it.
We just don't want the same matrix with different names out there as some already are ie., AUTO-Q1/AVAMAT6.

Thanks again for the information and matrix.

Happy New Year !http://users.pandora.be/eforum/emoticons4u/happy/143.gif

manono

31st December 2005, 08:36

Hi-

I think that the name Fox Home Entertainment came from someone that got it off of a Fox DVD, perhaps using ReStream, back before there was a Rockas Matrix Editor, or a way to extract them using DGIndex. It first showed up in the XviD Forum back during the custom matrix creation craze. So I don't think that's any sort of an official name or anything like that. As for how I name them, I use the lower right number of the Intra and Non-Intra matrices to tell them apart. So the Fox matrix I call 41/11, and that one I posted last time is the 16/16 matrix. Works for me. And I'm using the Fox version with the 14 for some of my own encodes with CCE, not that it makes any difference.

BTW, thanks for the New Matrix. Would you suggest using it at say around the 4800+ bitrate mark? (I know I'll try it on something and see for myself )

Although Rockas asked me to set some guidelines for which matrix to use and when, personally, I find that tieing a matrix to a bitrate to be pretty much useless. For me, it's a question of the average quant of the original, how much I'm compressing, and the matrix used on the retail. For example, the Fox Film Noir series DVDs use very high bitrates, have very low ave quants of 2-3, and use the 16/16 matrix. In addition, except for a commentary track and a trailer, they usually have no other extras at all. So I can use the 16/16 matrix with no problem. On the other hand, if a film uses the Standard matrix originally, with an average quant at 7 already, it doesn't matter how much or how little you're compressing it, as you're already pretty much screwed as far as using a decent matrix goes. But the Standard matrix has already wiped out so much detail, that I don't guess it makes much difference, unless you plan to try and sharpen it back up using LimitedSharpen, filter it, or some such.

Matrix discussions are still pretty new in amateur MPEG-2 encoding circles. I agree with you 100% that a wise choice of matrix can go a long way towards making your backup as good as possible. We're still pretty much on uncharted ground here, and I'm still learning as I go along. SAPSTAR is probably the matrix expert at Doom9, and knows more than anyone about them.

feedback

31st December 2005, 09:17

I was checking some of my other matrices and the method of naming you suggested may not work for some matrices. For instance, if the TmpegEnc CG matrix was called the 32/16 it would conflict with the Smooth CG matrix which is also 32/16 but which is quite different in many of the other values.

How about '16/16 High Detail' for the matrix you posted? It would be a way to give a little bit of a description of the matrix and also give it another identifier.

if a film uses the Standard matrix originally, with an average quant at 7 already, it doesn't matter how much or how little you're compressing it, as you're already pretty much screwed as far as using a decent matrix goes.

Yes, if I remember correctly you are not a big fan of the mpeg standard matrix and I see why if it kills the detail and leaves you little to work with.
High Quant leaves little wriggle room.

SAPSTAR is probably the matrix expert at Doom9, and knows more than anyone about them.

O.K., now I am going to have to check out SAPSTAR and his matrix knowledge. I have not seen much of him since he started his new encoder.

Regards,

manono

31st December 2005, 10:57

You're right about the naming convention not working in all cases. The Sony encoder matrices all have 127 in the lower right. For example there's this kick-ass high bitrate matrix:

08 08 08 09 11 13 14 17
08 08 08 11 13 14 17 18
08 09 10 11 14 17 17 19
09 10 13 13 14 17 18 127
10 13 13 14 16 17 127 127
13 13 14 16 17 20 127 127
13 13 14 17 19 23 127 127
13 14 17 19 127 127 127 127

08 08 08 09 10 10 11 11
08 08 09 10 10 11 11 12
08 09 10 10 11 11 12 12
09 10 10 11 11 12 13 127
10 10 11 11 12 13 127 127
10 11 11 12 13 13 127 127
11 11 12 13 13 14 127 127
11 12 12 13 127 127 127 127

And then there's this medium bitrate matrix which is in RME:

08 12 13 14 15 16 19 22
12 13 14 15 16 19 22 26
13 14 15 16 19 22 26 32
14 15 16 19 22 26 32 41
15 16 19 22 26 32 41 53
16 19 22 26 32 41 53 70
19 22 26 32 41 53 70 94
22 26 32 41 53 70 94 127

12 12 13 14 15 16 19 22
12 13 14 15 16 19 22 26
13 14 15 16 19 22 26 32
14 15 16 19 22 26 32 41
15 16 19 22 26 32 41 53
16 19 22 26 32 41 53 70
19 22 26 32 41 53 70 94
22 26 32 41 53 70 94 127

So I call them 127 High and 127 Low when taking notes and testing. I like that second one much better than the Standard or MPEG Standard matrices when using around 3000 bitrate.

Rippraff

31st December 2005, 14:11

Counting left to right as 0,1,2,3,4,5 etc. and down 0,1,2,3,4,5, etc., we look at the 3rd row 4th column the value is 13 in Fox but in 'High/High' Matrix the value is 14.
Not in my case. :confused:

Here's my original 1-High_High (4000+) matrix:

08 08 09 11 13 13 14 17
08 08 11 12 13 14 17 18
09 11 13 13 14 17 17 19
11 11 13 13 14 17 18 20
11 13 13 14 16 17 20 24
13 13 14 16 17 20 24 29
13 13 14 17 19 23 28 34
13 14 17 19 23 28 34 41

08 08 08 09 09 09 09 10
08 08 09 09 09 09 10 10
08 09 09 09 09 10 10 10
09 09 09 09 10 10 10 10
09 09 09 10 10 10 10 11
09 09 10 10 10 10 11 11
09 10 10 10 10 11 11 11
10 10 10 10 11 11 11 11

By the way thanks manono for the new one, as suggested by feedback I call it Fox 2. ;)

Cu Rippraff

manono

31st December 2005, 15:38

Maybe he meant the 5th column over, as that's where I see the 13/14 difference.

You're welcome, Rippraff.

archaeo

31st December 2005, 16:15

Matrix discussions are still pretty new in amateur MPEG-2 encoding circles. I agree with you 100% that a wise choice of matrix can go a long way towards making your backup as good as possible. We're still pretty much on uncharted ground here, and I'm still learning as I go along. SAPSTAR is probably the matrix expert at Doom9, and knows more than anyone about them.

Hey all, good discussion here. I have noticed the same thing - that good data on when and how to apply custom matrices is a little thin right now, at least for the level I understand them at. There certianly is a plethora of custom matrices out there now, but not a lot of information on the reasoning and application of them, beyond just 'bitrate'. I've been looking at this topic a lot more lately, spending much of my time reading previous posts to try to understand the process better.

Your comment manono about looking at the quant level, matrix, and amount of extras, etc.. of the original makes a lot a of sense to me. It isn't only about bitrate, but about several things like the quant factor and the matrix used on the original. This gives me a little more info to look at before deciding which way to go.

I think as we spend more time on these things, there will be more discussions to join in, like this one. This thread could be a good one to keep going. thanks for sharing your process :goodpost: Hopefully as I get more informed on this topic, I will be able to add something too.

feedback

31st December 2005, 20:59

Not in my case. :confused:

Here's my original 1-High_High (4000+) matrix:

08 08 09 11 13 13 14 17
08 08 11 12 13 14 17 18
09 11 13 13 14 17 17 19
11 11 13 13 14 17 18 20
11 13 13 14 16 17 20 24
13 13 14 16 17 20 24 29
13 13 14 17 19 23 28 34
13 14 17 19 23 28 34 41

08 08 08 09 09 09 09 10
08 08 09 09 09 09 10 10
08 09 09 09 09 10 10 10
09 09 09 09 10 10 10 10
09 09 09 10 10 10 10 11
09 09 10 10 10 10 11 11
09 10 10 10 10 11 11 11
10 10 10 10 11 11 11 11

By the way thanks manono for the new one, as suggested by feedback I call it Fox 2. ;)

Cu Rippraff

O.K. I was afraid there might be some confusion. I was using the XY axis method starting from zero, as found in LigH's 'Custom Quantization Matrix Editor'.
Anyway a picture is worth a thousand words so the value in question is now obvious below.

08 08 09 11 13 13 14 17
08 08 11 12 13 14 17 18
09 11 13 13 14 17 17 19
11 11 13 13 14 17 18 20
11 13 13 14 16 17 20 24
13 13 14 16 17 20 24 29
13 13 14 17 19 23 28 34
13 14 17 19 23 28 34 41

08 08 08 09 09 09 09 10
08 08 09 09 09 09 10 10
08 09 09 09 09 10 10 10
09 09 09 09 10 10 10 10
09 09 09 10 10 10 10 11
09 09 10 10 10 10 11 11
09 10 10 10 10 11 11 11
10 10 10 10 11 11 11 11

@manono

Does the 'Manono 127 High', that you just posted, give more detail than the Fox 2 (as Rippraff calls it)? That is If the quant is not very high?
Moreover, you had 6% improvement with the latest Fox matrix over the old Fox Home Entertainment matrix. What percent improvement do you see on the same or similar source.

Hey, why not use your nick as part of the description...Didee, Kika, Soul Hunter, and Sharktooth etc., do it. As far as description goes, with the 'High' part of the matrix name users will be given, maybe a little, direction in it's use. Also, the structure of the Matrix is somewhat described with the 127 designation. Kind of like 'Andreas 78er Matrix' where the Intra and Non-Intra values in the lower right are 78.

Happy New Year to All,:)

manono

31st December 2005, 23:26

I did a bunch of tests using a chapter from the R1 LOTR-ROTK-EE. I used a bitrate of 3000 and compared the resulting average quant of each. The results may be different for different material and for different settings in CCE. Using the Fox matrix as the base, assigning it a value of 100%, the 16/16 matrix came out at 106% and that 127 High matrix at 102%. Except for matrices generated when using AQ turned on in CCE (as several studios are doing nowadays, including New Line with LOTR-ROTK-EE), those are the only 2 I've come across with percentages higher than the Fox matrix. The Standard matrix came out at 45%. There's nothing wrong with a 45% rating, as we all sometimes need matrices that compress well. I just don't like how it gets there. It's possible I'm wrong about the conclusions, but it tells me the amount of detail retained relative to each other.

Someone like SAPSTAR has learned the ins and outs of matrices, so he can generate compliant matrices fitted to the material being run through CCAQM. Guys like Didee, Soulhunter, Kika, and Sharktooth also learned about matrices so they could develop their own. They did the work and they can call them whatever they like. They all know a great deal more about them than I. The matrices they developed (Didee, Soulhunter, and Sharktooth anyway) were developed for use with MPEG-4 (XviD). I have no interest in any of that. I've read up on them, but it makes my head hurt. It's pretty complex stuff. I just grab matrices off of DVDs and fool around with them to find a spread of different matrices to use with my own encodes. I figure the guys that developed them for use in hardware encoders know more about the subject than I ever will. So none of the ones I use (except for one I've modified from one I got from a DVD) are really mine in any sense of the word.

Rippraff

31st December 2005, 23:50

O.K. I was afraid there might be some confusion. I was using the XY axis method starting from zero, as found in LigH's 'Custom Quantization Matrix Editor'.
Sorry I've missed the zero, maybe because this isn't the way I'm counting. ;)

Happy new year to all of you, 10 min to go in Berlin... :)

Cu Rippraff

feedback

1st January 2006, 22:50

So none of the ones I use (except for one I've modified from one I got from a DVD) are really mine in any sense of the word.
I understand. So, 127 High it is.

I have now used the 127 High Matrix and the Fox 16/16 Matrix and the backups do look very good IMHO...with greater detail of course. :D

BTW, this is my first post of the new year.
Regards,:)

P.S. When trying to save the Fox 16/16 matrix in RME I get the following error.

Run-time error 76
Path not found.

The problem is when I use the / or slash between the two numbers.
When I use the - or dash it works fine.
So I saved it as Fox 16-16.

Rockas

1st January 2006, 23:18

@Feedback
Yes... that's right :)
You can't use the "\" character on file names... OS Obligue

Edit: you can't use any of the following characters:
/
\
:
*
?
<
>
|
"

Mr. Monte

2nd January 2006, 01:09

Except for matrices generated when using AQ turned on in CCE (as several studios are doing nowadays, including New Line with LOTR-ROTK-EE.

Thanks for that test...That's why I posted a while back about the CCEAQM switch in rebuilder. I too have been doing my own "visual" test and have found using the CCEAQM definately makes my backups look better than without. Especially older black & white series disc (Lost In Space Season 1)

archaeo

2nd January 2006, 01:50

Re: CCEAQM
Since the matrix adjustment is done entirely within CCE - Is the only way to see the modified matrix done by extracting it from a segment, and comparing it to the matrix chosen as the default? Has anyone taken a look at these AQM modified matrices? Just wondering to what level it adjusts the default matrix to fit the demand.

Mr. Monte

2nd January 2006, 04:21

My next question is if several studios are now using CCEAQM on their releases...would it be safe to say it is not as problematic with stand-alones as originally belived and published in the older CCE manuals?

manono

2nd January 2006, 08:23

Hi-

As I found out when SAPSTAR set me straight in your earlier thread, Mr. Monte, his CCEAQM is very different from CCE with AQ (or more accurately, CCE with the "Disable adaptive Q-matrix switching" box unchecked). Where CCE with AQ uses one matrix as a base, and then halves and quarters the numbers, CCEAQM when used together with DVD-RB, I think generates a specific custom matrix suited for each video segment. CCE with AQ can't be used directly in DVD-RB. Maybe it can with one of the other CCE plugins. I don't know as I don't use any of them. CCEAQM, of course, can be used together with DVD-RB.

So no studios are using CCEAQM with their DVD releases, but CCE with AQ and the Standard Matrix as the base. When encoding on your own with CCE, outside of DVD-RB, you can use the Standard matrix, or any other matrix you choose, as the base matrix.

As far as causing problems with standalones, since SAPSTAR has taken care to generate only compliant matrices with CCEAQM, they should play fine, especially since the switching takes place only every segment. CCE with AQ can switch as often as every GOP. Even though all DVD players are supposed to handle AQ with no problems, I guess that not all do. The CCE manual does have a caveat about CCE with AQ written in its own charming English:
However, some DVD player does not support such MPEG streams as encoded with adaptive quantization matrix, and may cause block noise when playback.
That said, I've never noticed any problems along those lines with my own encodes on my own or other players on which I've tested.

archaeo

2nd January 2006, 14:06

...CCEAQM when used together with DVD-RB, I think generates a specific custom matrix suited for each video segment...

@manono
have you ever extracted one of these custom AQM matrices from an encoded segment to look at the values it 'modifies'? I'm curious as to how much it tweaks the original matrice.

manono

2nd January 2006, 14:21

I wouldn't know archaeo, as I've never used it. Sorry.

Mr. Monte

2nd January 2006, 16:10

Thanks for a great explaination

Gibuk

3rd January 2006, 11:15

Thanks for your availability!
I imagine that the new Fox matrix gives turns out better than the old one ;)
I'm trying to compress the anime "Ninja Scroll": vbr bias=20, quality prec=? 16 would go well or too much low? Using Rme to extract its matrice, I has noticed that it is the common Mpeg standar matrice; if I used high high matrice (I have obtained bitrate over 5000 for the movie), I improve the compression?

manono

3rd January 2006, 11:55

Hi Gibuk-

Ideally you want the original DVD to have used a good matrix (Fox or similar), and then, depending on how much the original was compressed, and how much more you're recompressing it, you'll be able to choose a good matrix for your encode.

If you start with a mediocre matrix, I don't really see how you can improve on the video. Once that detail is gone, you can't really get it back. So in your case where the Standard matrix was used in the retail DVD, I don't think that using a good matrix will improve it, but it can maybe be used to keep it from degrading even more than necessary. If I remember correctly from having encoded Ninja Scroll years ago, it's not one of those anime with a lot of real detailed animation. One thing you do have to worry about when encoding anime is mosquito noise around the dark lines. This means, among other things, not finishing with too high of an average quant. I don't know if that answers the question.

I'll beg off answering the question about VBR Bias and QP. There are already quite a few threads about that with many opinions set forth. apfraats is involved in many of those threads.

archaeo

3rd January 2006, 14:57

Hello,
To those interested in how CCEAQM works, I just ran some tests with AQM activated on a backup of 'the thin red line', approximately 170 min, 69% compression, movie only, with an average bitrate of 2930.

First of all I extracted the original matrix from the original dvd w/ RME, which gave me this matrix (I'll call it TRL_1):

08 08 09 11 13 13 14 17
08 08 11 12 13 14 17 18
09 11 13 13 14 17 17 19
11 11 13 13 14 17 18 20
11 13 13 14 16 17 20 24
13 13 14 16 17 20 24 29
13 13 14 17 19 23 28 34
13 14 17 19 23 28 34 41

08 08 09 09 10 10 11 11
08 09 09 10 10 11 11 12
09 09 10 10 11 11 12 12
09 10 10 11 11 12 13 13
10 10 11 11 12 13 13 14
10 11 11 12 13 13 14 15
11 11 12 13 13 14 15 15
11 12 12 13 14 15 15 16

Attached is a screenshot (TRL_1) of this matrix captured through ligH's CQME. (Someone might recognize this one as an already identified matrix, but i haven't had the chance yet to see what that would be.)

Now, for the test, I took this original matrix and applied it to the encode using RB/CCE 2.70, 3 pass, AQM activated.

After running through RB/CCE, with AQM activated, I wanted to see how AQM modifies this original matrix to meet the demand. I figured with a compression level of 69%, it HAD to adjust the original in some way. So, after the encode I extracted the matrix again from the same area of the film, wanting to see how the AQM feature in CCE actually modified the matrix. The result is the 'adjusted matrix' (I call it TRL_2), as prepared by CCE's AQM feature:

08 08 08 08 08 08 08 10
08 08 08 08 08 08 10 11
08 08 08 08 08 10 11 12
08 08 08 08 10 11 12 14
08 08 08 10 11 12 14 16
08 08 10 11 12 14 16 19
08 10 11 12 14 16 19 21
10 11 12 14 16 19 21 24

08 08 08 08 09 11 12 13
08 08 08 09 11 12 13 15
08 08 09 11 12 13 15 16
08 09 11 12 13 15 16 17
09 11 12 13 15 16 17 20
11 12 13 15 16 17 20 22
12 13 15 16 17 20 22 25
13 15 16 17 20 22 25 25

As you can see, the new matrices are significantly different, particularly the the intra frame matrix (See attached TRL_2 screenshot). It's more than just a few changed fields, as I would have suspected. Someone with better knowledge than I of matrices may be able to analyze these and tell a bit more about the differences between them (?). I haven't had the time yet to do side by side quality comparisons between the original and the encode, but a quick look seemed to show that the resulting quality is very good.

During the encoding process, I did notice that the encoding time and progress bar for particular segments of the film using this original matrix was long, noticeably longer than when using the standard MPEG matrix. I am guessing that CCE was using this extra processor time in calculating the modified matrix. I assume that if I had chosen a matrix that offered more compressability than the original (like the MPEG standard), AQM might never have been activated, and the encode time would have been less.

I think that if you decide to encode with an original source matrix, especially if it is not very compressible, AQM is the way to go to adapt to the demands of the compression.

Mr. Monte

3rd January 2006, 23:58

I could be mistaken...however, I thought jdobbs told me in another thread that the CCEAQM option activated...disreguards any matrixes selected.

archaeo

4th January 2006, 00:12

From my understanding, AQM is an 'adaptive' quantizer matrix feature. According to CCE:

"There are some cases that Cinema Craft Encoder SP cannot keep
the minimum bitrate. It is considered to be caused by that the lowest quantization scale cannot raise the bitrate under a single quantization matrix. To solve the problem, Cinema Craft Encoder SP automatically creates other matrices cutting the value of each element in the matrices by half, quarter and so on, and applies another one to produce a much lower distortion for much higher bitrate. This switching of quantization matrix can occur at every GOP."

I think with AQM enabled, CCE will run the chosen matrix through the encode UNTIL it needs to adapt it to meet the compression demands...What I noticed on my test was that it did in fact keep the TRL_1 matrix at certain segments, but also adapted it to meet other segments by creating it's own matrix, TRL_2.

Mr. Monte

4th January 2006, 00:43

Makes sense..however..if when needed CCE will cut the matrices..why would anyone start out with a low bitrate matrix when using this option (CCEAQM)...it would seem better to start out with a high bitrate matrix and just let CCE adjust when needed

Am I on the right track?

archaeo

4th January 2006, 01:35

Yes, but there may be some benefit to choosing at the outset a good compression (or other) matrix that you know rather than letting CCE 'adapt' to one. I have limited knowledge of the matrices, but if I choose one that trims the bitrate in the way I want it to, it leaves less to chance that CCE may cut bitrate where I don't want it to. Choosing a high bitrate matrix with a source that needs big compression, has low bitrate, etc.. will probably guarantee that CCEAQM will modify it, but perhaps not in the way that you want it to.

manono

4th January 2006, 17:52

archaeo

4th January 2006, 21:19

archaeo-

That first matrix you posted (TRL_1) is commonly used in commercial DVDs, perhaps more so several years ago before the rise of the Fox matrix, but it's still pretty common. It's included in the Rockas Matrix Editor as 2. High_Medium. By my tests it retains about 94% of the detail when compared to the Fox matrix. I use it quite a bit in my own encodes. It's also very similar to the Standard matrix with the numbers halved, and shows up when you use CCE with AQ turned on.

The one generated by CCEAQM (TRL_2) is actually a sharper matrix than the original. I ran it through my test chapter from the R1 LOTR-ROTK-EE and it got a 99% figure (compared to 100% for Fox).

I don't know what any of that proves, but I thought you might like to know. If you still have the MPV segments, I'd be curious to know what the ave quant of that section was.

Virginia, eh? I grew up in Arlington. You from the DC area also, or from the "real" Virginia farther south? The 'Skins made it into the playoffs. Yippee!

hi manono,

hey, thanks for the information. I do find it very interesting...
I thought I had seen the first matrix before, so that explains it. I find it perplexing that the TRL_2 ended up being a sharper matrix - and therefore less compressible? I would have thought that AQM would have taken the original matrix and made it more compressible, less sharp in order to get it down to size :confused: I do not have the MPV segments (do you mean the video files created by CCE prior to Rebuild?) any longer. However I may run it again to see what the quant ends up being - by the way, what do you use to get the most accurate quant reading? I have bitrate view, but is this the recommended progggie for pulling that data out?

No, not from Virginia originally, moved here for work about a year ago. I'm glad to see the 'skins make it - I'm rooting for them unless thay play the bears - I'm from the windy city ;) )

manono

5th January 2006, 07:53

Hi-

I use the free version of Bitrate Viewer. It's supposed to be accurate for PAL, but I don't know for sure. For NTSC it seems to come out about 15% higher than what you see when checking the quant graph in CCE, or checking the max bitrate against what was set in CCE. A guy at videohelp.com said that it doesn't take pulldown into account, but that can't be true, because if you run an MPV, an M2V, and a Vob of the same video through it, the figures it spits out are identical or nearly identical. I don't care much about the exact figures for my tests anyway, but the relative figures, which I hope are accurate. I know that if it says, for example, 7, that it's really closer to 6.

I might have to test out some of the other programs suggested to check the bitrates.

jdobbs

5th January 2006, 14:40

I looked at it briefly. I think it always shows you the playback bitrate rather the the bitrate as stored on the disc -- so telecined sources could show a bitrate 25% higher than what is actually stored on the disc.

Gibuk

5th January 2006, 16:16

One curiosity: if I exctrated the matrix from a any movie (for example Batman Begins), by Rme, and I applied it during the compression of the same film (keeping high bitrate for only movie), I would obtain the better result as quality or would be more useful to use High_High or new Fox matrix?

manono -

about Ninja Scroll, if I used the new Fox matrix in place of the High_High, I would get something in terms of quality?
Thanks

manono

5th January 2006, 16:52

Hi-

about Ninja Scroll, if I used the new Fox matrix in place of the High_High, I would get something in terms of quality?

There's not much difference between those 2 matrices, certainly not enough for your eyes to tell them apart. There is a big difference between those 2 high bitrate matrices and the Standard matrix used in the retail version of Ninja Scroll. So I think I would answer "no" to your question.

archaeo

6th January 2006, 01:29

If you still have the MPV segments, I'd be curious to know what the ave quant of that section was.

manono,
FYI: I still had the segments for this test and went ahead and ran the test segment through bitrate viewer.

encoded average Q level: 7.35,
encoded average bitrate: 2735.

(edit)
source average Q level: 3.41
source average bitrate: 4352

Seems to be a pretty reasonable Q, so I wonder if this is th reason why AQM created such a 'sharp' matrix in TRL_2?

manono

6th January 2006, 06:27

Yes, the average quant is very reasonable. As for the reason that matrix was chosen, I guess you'd have to ask SAPSTAR. Maybe the whole film compresses well, or that's just a fairly static segment.

archaeo

8th January 2006, 16:57

I'd like to ask another question to get a better handle on this issue of applying custom matrices to RB projects. From what I understand, choosing a matrix just on average bitrate may not be the best way to go. I'm starting to look more at the resultant Q level as well as other things.

I've started to use the following method to determine if and what custom matrix I might apply on the encode:

I create a small but representative sample clip from the source (approx 100-200 MB)

I run this clip through RB (CCE) using several different matrices, say, from a high detail less compressible matrix like Fox Home entertainment, all the way down to more a more compressible matrix like bulletproof's heavy compression matrix

I then run each separate clip through Bitrate Viewer, taking notes on average Q level, average bitrate, overall visual quality.

As an example, I had the following results on a test clip
taken from 'Andersonville', a predicted 62% compression level:

Original Source test clip:
average bitrate: 4909
average Q level: 7.81

Angel best very low:
average bitrate: 5017
average Q level: 3.17

Bulletproof heavy comp:
average bitrate: 4931
average Q level: 4.32

Fox Home Enter:
average bitrate:4964
average Q level: 7.66

Question: I know a matrix like bulletproof's or angel's discards a huge amount of the high frequency stuff (detail), but from my visual check, it still looks OK. So, if that is the case, would you tend to choose a matrix that offers the LOWEST Q (angel), or, one that has the CLOSEST Q to that of the original source (like Fox)? My initial reaction would be to go with the matrix that holds the most detail compared to the original (and stays close to the original Q) , but I also have heard that getting that Q level as low as possible is preferable.

thanks for any feedback

manono

8th January 2006, 18:07

Hi-

...but I also have heard that getting that Q level as low as possible is preferable.

Following that line of reasoning, you might as well use CCE's Ultra Low Bitrate matrix, or filter the heck out of it with Convolution3d (preset="vhsBQ") or some heavy duty smoother, but you won't like the results, even though you'll wind up with a nice low ave Q. No, going only by the ave Q of a full bitrate encode without taking into account the matrix used and any filtering done is a red herring. And how much will you be compressing the source?

You're getting into an area where I don't have much more to go on so far besides my gut instinct. Here's another way of asking the question. If the Standard Matrix retains only 45% of the detail when compared to the Fox matrix, is the Standard matrix with an ave Q of 3 the rough equivalent to the Fox matrix with an ave Q of 7? I submit that it's not. It's much worse. Why? Because the Standard matrix starts out removing so much detail that the Fox Matrix will have to go to a much higher quant than 7 to reach that level of detail removal. In addition, the Standard matrix, besides removing high frequency detail, has also removed quite a bit of the much more obvious low frequency detail. Except for the first 8 in the Intra matrix, its numbers start at 16 in the upper left. and go up rapidly from there, particularly the Intra matrix.

And comparing paused frames or frame captures isn't always a good way to compare matrices. It's just too hard to spot the differences. I've had some luck capturing stills and then blowing up a portion of them to 5-600%. What you want to do is to compare how different the reencode is from the source, how much detail has been removed from the source by reencoding it using different matrices at different ave quants. We know that just the act of reencoding, no matter how high quality the settings, degrades the source to a significant degree. But will using different matrices really make a difference? You want to see an obvious difference? Use CCE's Very Low Bitrate matrix, or Jawor's or some other low bitrate matrix, and compare it with the Fox for a similar bitrate. You already did that and said you didn't see much difference. Blow up the same portion of each of them and run them through the Windows slideshow viewer. Then when you see them one after the other, the differences are much more obvious. But you should see at least some difference in the sharpness and complexity of detail even at full size. If you're watching on a standard interlaced TV set, that difference might not be noticeable. If you watch on a big screen HDTV, that difference will be much more obvious.

Here's something I plan to do one of these days. You want to magnify the differences, to make them more obvious. Here's a script that will do it. I got it from the AviSynth scripts collection at Soundforge:
To make the differences between original and reencoded video more visible.
# If Videos start at different frames
frameadjust=0

# Videos to compare: (v1 is original, v2 is encoded or whatever)
v1 = MPEG2Source("original.d2v",false).trim(frameadjust,0)
v2 = MPEG2Source("encoded.d2v",false)
sub = v1.subtract(v2)
substrong = sub.levels(122,1,132,0,255)
I'm quite certain that running that script on 2 of your encodes will make the differences quite obvious (he says, not having done it yet himself).

archaeo

8th January 2006, 21:01

...Here's another way of asking the question. If the Standard Matrix retains only 45% of the detail when compared to the Fox matrix, is the Standard matrix with an ave Q of 3 the rough equivalent to the Fox matrix with an ave Q of 7? I submit that it's not. It's much worse. Why? Because the Standard matrix starts out removing so much detail that the Fox Matrix will have to go to a much higher quant than 7 to reach that level of detail removal. In addition, the Standard matrix, besides removing high frequency detail, has also removed quite a bit of the much more obvious low frequency detail. Except for the first 8 in the Intra matrix, its numbers start at 16 in the upper left. and go up rapidly from there, particularly the Intra matrix.

Yes, I can see the logic there on not going too much with Q as a decision making tool. The fact that the heavier compression matrices discard so much at both the lower and the higher frequencies makes Q level somehat irrelevant in this case, especially if the original source matrix is one that has 'held' quite a bit of detail (like Fox)...

And comparing paused frames or frame captures isn't always a good way to compare matrices. It's just too hard to spot the differences. If you're watching on a standard interlaced TV set, that difference might not be noticeable. If you watch on a big screen HDTV, that difference will be much more obvious.

Point taken. Seeing them not only in a large format view, but also IN MOTION will probably give me a much better idea of how well those matrices are working on those predictive (P) frames. At this point I have been viewing primarily on my 19" monitor, and blowing them up as stills. I think I will put these clips on a DVD, and look at them on my 42" ED to see how high-motion or other challenging scenes are dealt with.

Here's something I plan to do one of these days. You want to magnify the differences, to make them more obvious. Here's a script that will do it. I got it from the AviSynth scripts collection at Soundforge:

To make the differences between original and reencoded video more visible.
# If Videos start at different frames
frameadjust=0

# Videos to compare: (v1 is original, v2 is encoded or whatever)
v1 = MPEG2Source("original.d2v",false).trim(frameadjust,0)
v2 = MPEG2Source("encoded.d2v",false)
sub = v1.subtract(v2)
substrong = sub.levels(122,1,132,0,255)

I'm quite certain that running that script on 2 of your encodes will make the differences quite obvious (he says, not having done it yet himself).

Yes, having some help in 'magnifying' these differences would make it a lot easier to see them- it gets tough looking at still after still, or running test clips right after one another to try to 'catch' the differences. Is this a script that can be inserted into RB's filter editor?

By the way manono, thanks for the response- it really helps to bounce these thoughts around

manono

9th January 2006, 03:16

Hi-

Is this a script that can be inserted into RB's filter editor?

I don't know. But you cut a piece of a Vob for your tests, right? So, make D2Vs from your various reencodes and use that for the comparisons.

One thing I didn't quite understand. You said you'll have 62% compression (Did I only now notice that, or did you edit it later on? Probably another case of me reading carelessly.). I think you're doing your tests using the original bitrate, rather than the lower reencoded bitrate, but I could be wrong. When being recompressed, a low bitrate matrix will deteriorate in quality much more quickly than will a high bitrate matrix. If the original bitrate is 5000, I think you'll get better results using 3100 for your tests, in order to give you a idea of what the final results will be. You may also find that Fox isn't the ideal matrix to use if the ave quant goes too high. You may have to step down to a slightly lower bitrate matrix. I know, it won't be exactly accurate, as DVD-RB will assign different bitrates to the different segments, but it should give you a better idea of the final results.

Oh, and I left out an important line in that script before:

# If Videos start at different frames
frameadjust=0

# Videos to compare: (v1 is original, v2 is encoded or whatever)
v1 = MPEG2Source("original.d2v",false).trim(frameadjust,0)
v2 = MPEG2Source("encoded.d2v",false)
sub = v1.subtract(v2)
substrong = sub.levels(122,1,132,0,255)
return StackVertical(StackHorizontal(v1.subtitle("original"),v2.subtitle("encoded")),StackHorizontal(sub.subtitle("Difference"),substrong.subtitle("Difference amplified")))

The script is here:

http://www.avisynth.org/SeeTheDifference

And here's one that sticks 4 different videos in a square so you can go over them all at the same time:

http://www.avisynth.org/BoxCompare

Lots of nice scripts here:

http://www.avisynth.org/ShareFunctions

Gibuk

9th January 2006, 11:43

sorry but.. how and where can I apply Seethedifference script?

archaeo

9th January 2006, 13:10

manono,
Yes that is the case, I did not reduce the bitrate to a level that would be found during the actual encode. How do you force the bitrate to a particular level? Is RB Opt one of the ways to do this?

manono

9th January 2006, 15:47

Don't know. I've never used RB-OPT. I've never used SAPSTAR's CCAQM. I've never used DVD-RB for any of my testing. I've never even used the RME for this kind of testing. I don't know anything about using the filter editor, or anything else within DVD-RB for testing such things. So:

How do you force the bitrate to a particular level?

You make a D2V from your original sample. You create an AviSynth script. You open it in CCE, set the bitrate and everything else and reencode several times using the matrices you wish to test. Then when done, make new D2Vs for your various samples that used different matrices, and put them in the script. You can compare the original against your various samples, and the various samples against each other.