Custom matrix with low to medium bitrate...need advice!

[anaqim]

Hi,

Been reading, testing and studying alot regarding the x264 encoder but this is my first post.

The reason I post is because I cannot seem to find answer to a relatively simple question for you guys.

I've done my test and my conclusion regarding encoding DVDs to x264 (AVC) is as follows:

I am using STAXRIP as my GUI simply because am aiming for a specific quality and not a specific size. Also it feels easier than megui.

My encoder settings are:

2-Pass encoding
B-frames = 3
Reference frames = 5
Loop filter on and -2/-1

Direct mode = Auto
Bias = 0
Adaptive = ON
Pyramid = ON
RDO = ON
Weighted prediction = ON
Bidirectional ME = ON

Scene cut = 40
CABAC = ON
GOP Size = 25/250 (mostly doing PAL)

Mixed Refs = ON
No fast P skip = ON
Motion estimation method = Multi HEX
Subpixel Motion Estimation = 7
Trellis RD Quant = 2 (Always)

P4x4 = ON
I4x4 = ON
P8x8 = ON
B8x8 = ON
I8x8 = ON
8x8 DCT = ON

Show progress indicator = ON
Disable PSNR computation = ON
Threads = 4 (X3800+)
Output type = MKV

Custom Matrix = Soulhunter V2


What I do is I CROP the movie and set it up without any filters except a SHARP resize to 720x resolution.

Then I do a 10% compressibility check first and aim for a quality of 60%.

I find the output quality on screen to be adequate for watching movies without getting irritated by artifacts.

For sound I use OGG -0 (approx. 64 bitrate) in stereo.

I only recently started to use the custom matrix and so far it seems it helps with the sharpness a little tho it seems it increases the bitrate some.

MY BIG QUESTION IS:

I know that sharktooths custom matrix is for bitrates a little higher than what I seem to be getting @ 60% so am I doing right using the soulhunter v2 matrix? would it be better to use the default flat one? or is there perhaps somewhere else a better matrix for movies with bitrates that often end up in the range of 600-1200 ?

ANAQIM

[elguaxo]

For Low Bitrates I use this one: http://forum.doom9.org/showpost.php?p=787311&postcount=19

[anaqim]

Thx, I tried it but the quality it produces is a bit too rought for my taste even compared to the built in flat matrix which by the way is understandable as the filesize shrinks somewhat.

anyone else got a suggestion?

[Sharktooth]

threads 4?
do you have a quad core CPU? Lower it to 2 (supposing you have a dual core).
However did you try my matrix at your bitrate?

[anaqim]

Yeah, dual core and i dont know why but i seem to get a minor increase in speed using 4 instead of 2 threads...maybe there is some hypertransport on them but then i thought that was an Intel thingy.

Yes i tried your matrix and i like the result though the file size and bitrate does increase some (10-20%) compared to the default matrix.

I read somewhere that your matrix is optimized for higher bitrates which is why im unsure about using it for my 600-1200 typical bitrate movies...might be a little higher with your matrix tho.

So, would you with your knowledge about matrixes (i know nothing about them) say that I should be able to use yours and get good results at these bitrates or would you recommend another one, or even the flat one?

Thx

Anaqim

[Sharktooth]

4 threads are useless on dual core. Use 2.
My matrix uses unbalanced coefficients to avoid blocking, that leads to a lower compression (and consequently higher bitrate).
If you use a 2 (or more) pass RC mode you can constrain the rate-control to converge to your chosen bitrate so there should be no problems in hitting the desired filesize.
The drawbacks will be the higher quants forced by the rate control, but you should do some tests and see if the resulting quality will be acceptable at your desired bitrate.

[anaqim]

now you are talking stuff i dont know too much about yet but if i understand this correctly, your matrix forces in some technical way a higher threshold which gives a higher quality.

which means that when i use it and do a compression check (10%) and then set target quality to 60%, quality will still be 60% like with the flat matrix only that the extra bits (obvious from the filesize) are forced in where needed to improve the picture the way your matrix was constructed to do it...

dont know if this makes much sense or if im way off track :)

Anaqim

PS. note that I use staxrip and the function (compression check) on 10% of the material to decide quality/bitrate

[Sharktooth]

i dont know exactly how the staxrip compression check is done, but i suppose it uses constant quantizer to do it.
if so the compression check should return a lower value for quality (due to the bigger filesize) but please ignore it and check the results with your own eyes.

[anaqim]

the compression check command line is:

--crf 18 --ref 5 --bframes 3 --b-pyramid --weightb --b-rdo --bime --direct auto --filter -2:-1 --analyse all --8x8dct --no-fast-pskip --subme 7 --me umh --trellis 2 --mixed-refs --progress --threads 4 --no-psnr --cqmfile f:\data\settings\eqm_avc_hr.cfg

when i do this test the resulting compressability is very different with and without the --cqmfile parameter.
when it is with, your matrix its always the higher bitrate required to obtain the desired quality (60%)

would i then not be able to assume that the "check" uses the matrix in a proper way to return a relatively realistic value?

[Sharktooth]

no... it's a normal behaviour coz my matrix produce bigger files at a fixed quantizer so staxrip assumes the source is less compressible (while it's the matrix that lowers the compression).

[anaqim]

hmm....

ok so the file is less compressable due to your matrix.

1) either that means everything is fine and I should select 60% quality and thereby accepting that the larger filesize is needed to retain quality due to the requirements of your matrix...

or...

2) the matrix makes the precheck serve me not real values regarding the compressability/quality so i should then perhaps set the quality to 50% believing that its similar to 60% anyway

I know you prefer megui where there is no such thing as a compression check but which alternative seem more logical to you? my impression is the first.

thx!

[unmei]

to compare the two matrices, you usually would want to hit the same filesize for both.

With 1) you don't know whether it was actually the matrix that helped you, or if you would have gotten even a better result with the default one but aiming for that higher file size.

PS wth is a "real" compressability.. That is a super-vague term (almost in the "quality" and "best" category ;)) and if you took your 60% preference over from a xvid-based program you can count yourself lucky if staxrip does the check for x264 in a way that is "comparable".

[*.mp4 guy]

In order to get a good idea of which matrix you like better the encodes you compare have to be from the same source, with the same settings and the same bitrate. That way you can be sure that the only thing introducing differences between the two encodes is the matrix.

My low bitrate matrix softens the picture (rather noticibly, unless you change the inloop settings) because at low bitrates I would rather have a soft picture then artifacts, if your looking for a sharper picture at low bitrates you shouldn't use it.

[anaqim]

Allright, starting to see some light here.

i'll make 3 encodes of a part fo the movie.

1 using the flat matrix, 1 using sharktoots matrix and 1 using souls v2 filter...all 3 at identitcal bitrate to compare which gives the best, to my eyes, result.

yes, mp4, im after a more sharp than smooth picture so i've already ruled out your low bitrate matrix.

as for the determination of compressability yes i agree, im only looking for a somewhat standard routine of doing these encodes.

i've made extensive tests on various clips including "saving private ryan" and "the matrix", both which are quite hard to encode detail and motionwise. All was done with the default flat matrix and I used my eyes to visually find the "quality" lvl that I agree with (best compromise). 50% was close but there were artifacts that got visibally much better at 60%. Then 70% and 80% didnt really give a noticably much better picture than 60% so thats where i landed.

When i find the matrix i want to use i'll make again some encoded on those hard clips comparing the 50%, 60% and 80% quality lvls to find my subjective optimal routine for converting DVDs to AVC.

Just for the record, im not doing any AC3 sound codings as they just take too much space. I still to the simple yet amazingly efficient OGG -0 quality which yields a VBR stereo track around 64 kbps. Naturally these movies are mostly for watching on laptops and normal screens. Always from HD and never burned to any discs.

I'll post my final results in case someone are interested.

Anaqim

[Sharktooth]

I would reccomend 48kbps AAC-HEv2 (or if you prefer, AAC+ v2) with Parametric Stereo for normal TV and Laptops coz the audio system isnt good enough to justify more quality.

[shon3i]

Maybe is better 48kbs without PS because last test on HA.org is prove that on 48kbs PS not be good, but for everything elese under is, so better to use HE-AACv1.

@anaqim what is source audio, stereo, 5.1 or?

because you can keep original 5.1 audio with very small size, you can use 96kbs and that will be fine.

[anaqim]

the trick is the AAC doesnt seem to work so well with staxrip as staxrip isnt really being updated. also I am using the matroska container which doesnt allow AAC sound i believe.

usually the sources are 5.1 yes. how can i keep it at 96kbps and it will still be fine? i need some help in software, settings and setup here i think. again, if it would be possible to do using staxrip then great :)

im not sure which container is the better so here im in the open....matroska or mp4...

p.s. test in progress

[elguaxo]

The nice thing about Matroska is that it supports everything, AAC included.

[Sharktooth]

well, if you go with AAC then MP4 is a good container, otherwise MKV is excelent but not supported by hardware devices.
oh, and MKV fully supports AAC too.

[shon3i]

usually the sources are 5.1 yes. how can i keep it at 96kbps and it will still be fine? i need some help in software, settings and setup here i think. again, if it would be possible to do using staxrip then great I newer use StaxRip so i don't know how to configure it, but probably he can compress to 5.1, btw you can use MeGUI to compress audio, with CT or Nero encoder to 96kbs keeping original channels

[anaqim]

i figured out how to get staxrip to encode aac from ac3 and i updated to the latest neroencodeaac file. gonna do some listening tests when i get home from work but i think the big decision will be if i should do 2ch or 6ch encodes. 6ch is flattering but will i ever use it? i'll need 6ch output from the machine i run it on which i doubt i'll bother to setup the system for. i have a full DVD home cinema setup but this doesnt allow 6ch external inputs so im hesitating about the usability for me to code 6ch.

my home cinema system also dont support aac as far as i know and also anyway i'm not going back to burning discs (that anyway deteriorate over a relatively short time)

matrix test has been encoded but again, i have to be home to check it out :)

Anaqim

[anaqim]

Did my visual and listening tests as intended and to make a long story short...

Staxrip using deblocking -2/-1 and Sharktooth's mid-high bitrate matrix gives, very subjectivly, the better picture.

For those who know staxrip, i do a compressability check first (10% of the movie instead of the default 5%) with the sharktooth matrix filter on. Once that has finished I select the 60% quality lvl which gives me exactly what i want in quality ;) The movie will be larger than when using the default flat matrix BUT the image is way better and the deblocking nature of sharktooth's matrix works wonders on the movies.

As for audio, Nero AAC with vbr quality 0.20 will give a stereo track around 48 kbps and it sounds clear to me (more than adequate for movies anyway)

Thanks to those who helped me!!!

If anyone using staxrip would like a simple solution on how to make good DVD 2 AVC rips I can always zip the folder including the codec preset and include some simple descriptions on how to achieve this. Just let me know.

Cheers!

Anaqim