Compare encode at same crf and test new AQ modes

[jpsdr]

Hello.

I'm intending to do some tests, but i want to verify if what i'm doing make sense.

I want basicaly to compare/test some AQ modes. Either just modifiying a bit an AQ parameter (strength if 1rst place), or compare new experimental AQ patch i've just discovered, even if they seems to exist since a little time (basicaly tmod x264).
I don't have time to watch carefully 15 times the same 25mn episode, so, i've thing of a first rough approach. As these new experimental aq mode patch were originaly made for increasing anime quality encode, my tests are made on an anime episode.

I'll do several encodes with the same crf (13 actualy), and between encodes, either changing the AQ mode or changing the aq strength a bit or using one experimental version. Not changing anything else.

I wanted to know, if the size of the result file can be a rough first estimation of the quality of the parameters or aq version used ?
Of course, it doesn't replace viewing/comparing, does it make sense (i mean, not totaly stupid), to say that if encode A gives a 1GB file, and encode B gives a 1,4GB file, the A way is probably more efficient thant the B way ? As files are suposed to match the same quality criteria, my first thought would be to say that if B method needs more bit than A method to achieve the same result, A is more efficient. Even more if one of the purpose is targetting Blu-Ray => size fixed.

They is of course several things i'm aware of :
- In range grade 0-20. An encode with average of 16 but min of 14, may finaly look better than an encode with average of 18 but a min of 2. Because even if it's on one frame, the 2 will be the only thing it will be remember, and it's screwing up the video. This can't be detected by file size. But, is this situation possible, in the test cases i've explained ?
- Result theoricaly are only for this file. Can be extended on similar files, but if nature of source change too much (not anime, grain/noise, etc...), result may not be the same.
- 8bit and 10bit may have different (even opposite ?) results. I intend to do all the tests for both 8 and 10 bits.

So, any enlightement an answers are welcome.

----------------------------------

The main purpose of these tests are for the new aq methods i've discovered in the tmod version of x264. Apparently, these patched were made by japanese people to increase quality of anime encoding. But, they lack the user manual !
How are they used ? Separately ? Together ? What parameters are generaly advised for specific kind of footage ?
These patched were made to try to improve things, but it's easy to screw up things when you don't know how handle parameters, and so finaly think it's not interesting.
So, i'm interested in any feedback it can exist on these patches, and feedback on both 10 and 8 bit, because as said, one on my purpose is targeting Blu-Ray format.
Maybe these patches are useless in 10bit but give good result in 8...? Who knows ? Not me, it's why i'm asking if there is people who know.

[jpsdr]

I begin to have some results, and when i saw them, i've trouble to realy understand the true meaning behind them.

I've made an encode with tune animation. This reduce the aq-strength from 1.0 to 0.6. Result file was 1.2GB.
I've redo the encode with the exact same parameters, but just putting the aq-strength at 0.8, in the middle. Result file was 1.7GB.
(It was crf=13).
Seeing the difference in size, i don't realy understand what it means. Is the first encode realy more efficient (as quality is supposed to be the same), or is there others things also ?

[MasterNobody]

1) Comparing quality of files with different bitrate / size is MEANINGLESS
2) Comparing size of files of different quality is MEANINGLESS
3) Comparing size of files with same crf and different other settings is DOUBLE MEANINGLESS because they are not equal in any of the objective metric not in bitrate / size.

REMEMBER, equal CRF do NOT mean that you will have equal quality (CRF don't have anything todo with quality other than lower value with same other settings will give better quality and higher value - lower quality).

So your comparison is broken from start. If you want comparison with at least potential to be meaningful read How to cheat on video encoder comparisons (http://x264dev.multimedia.cx/archives/472) and don't make same mistakes.

[jpsdr]

Thanks for these informations. I'm opened to any advices of how i can (if it's possible), make comparisons, because it's a allready of course a begining and a good thing to know what to not do, but in that case, i've still no ideas of what to do.

[LoRd_MuldeR]

Well, if you which to compare AQ modes, I would suggest you make several 2-Pass encodes with the identical target file size. This way you get files of the same size, whose perceived quality can be compared. And the only option that differs between those encodes should be the AQ mode. Or the AQ strength. But you probably do not want to change both at the same time. Also you need to pick a "reasonable" bitrate for the test. And here "reasonable" means that the bitrate should be low enough to actually see some differences between the encodes (it doesn't make much sense, if all encodes look equally great), but also not too low (it doesn't make much sense either, if all encodes look equally messed up).

Alternatively you could figure out the CRF values where the different AQ modes to be tested give the same perceived quality, so you get files of identical quality and can compare the file sizes. But that's much more difficult to accomplish :eek:

[jpsdr]

Alternatively you could figure out the CRF values where the different AQ modes to be tested give the same perceived quality, so you get files of identical quality and can compare the file sizes. But that's much more difficult to accomplish :eek:
That's what i (apparently wrongly) thought i would have encoding with the same crf value... :(
That's the purpose i was trying to achieve, but "automaticaly", whithout having to watch the results, time i don't have, as explained.
It seems that what i want to get is not possible...

[benwaggoner]

Actually, I think CBR encodes would be a more useful first step here, so we see how x bitrate does for each local bit of content, instead of how it moves bits across the whole title.

[jpsdr]

What x264 target when you ask for a specific crf ?

I mean, if i target an average bitrate of X, i know i'll have a file of a specific size. If i encode with parameters A asking/setting for bitrate X, and re-encoding the same file changing one parameter, so encoding with parameters B asking for bitrate X, i know that in both case, i'll have a file of the same size. Quality may differ, local bitrate repartition may differ, but average bitrate will stay the same (and so the size).

If i encode with parameters A asking/setting for crf X, and re-encoding the same file changing one parameter, so encoding with parameters B asking for crf X, in both case, i've asked x264 to achieve/reach/"i don't know what" the same criteria which has the value "X". This value X is uses somewhere to do something, and so, my both files should have "something" which is the same, because i've asked this "something" to have the same value : X.

I've been already told that it's not the quality, so, my question is : What is this "something" ?

[MasterNobody]

It will use constant rate factor (CRF). Where rate factor is ratio (dependency) of predicted frame complexity and bitrate (quantization parameter).

[jpsdr]

Thanks, for your answer, it's not obvious for me, but i hope i understand engouh, so this lead to another question.

predicted frame complexity : Is this always the same for the same file, or is it related to the encode parameters used, and can change even for the same file ?
If it can change for the same file according the parameters used :
- Has all the parameters the same effect/weight, or there is some with more weight/effect ? (for the same file)
- If there is some with more weigth/effect, which are the most important ? (for the same file)

:thanks:

[MasterNobody]

Both predicted frame complexity and bitrate (quantization parameter) depend from used options. I can't say which options have larger effect and which lower. But for example mbtree / qcomp / aq / b-adapt / bframes / psy-rd / subme have some effect so I would say it mostly all options that significantly change encoding process.

[jpsdr]

Ok, thanks.