[x264] Should I use AQ or Trellis?

[Bleck]

Hello back guys, I have been reading thousands of guides about this little issue, and I cant find out the answer.
It is worth to use --trellis 2 and --aq-mode 2 at the same time?
Or I should use one alone for best encoding?
:thanks: in advance!

This is the command line Im using actually (Similar to veryslow preset, I tried to use the commands the best, from guides. I probably miss some parameters...):

x264.exe --demuxer avs --output-csp i420 --threads auto --crf 27 --bframes 16 --b-adapt 2 --ref 16 --aq-mode 2 --qcomp 0.50 --direct auto --weightp 2 --8x8dct --partitions all --mixed-refs --me umh --merange 32 --subme 10 --trellis 2 --no-fast-pskip --rc-lookahead 50 --output "C:\Users\Developer\Desktop\video.264" %1

[Groucho2004]

As far as I know, aq-mode 2 is still experimental so I would stick with the setting used in the slow presets (1). Feel free to experiment.

Anyway, at CRF 27, it's going to be a blocky and blurry mess no matter what magic command line you construct.

[LoRd_MuldeR]

AQ (Adaptive Quantization) is always a good thing to have!

As x264 has AQ mode "1" enabled by default, you don't need to set "--aq-mode" explicitly, unless you want to try mode "2".

According to the changelog:
[--aq-mode 2] Generates significantly better SSIM; on by default with --tune ssim. Whether it generates visually better quality is still up for debate.

Trellis 2 is something that you would enable if you have the time for it (default is Trellis 1). x264 uses Trellis 2 with the "slower" Preset and above.

Generally I would recommend to just stick with the x264 Preset system and pick the slowest Preset you can accept speed-wise...

BTW: Do you have any specific reason to mess with the "--qcomp" option? It's one of the things that usually should not be touched by the users.

(And "--thread auto" is redundant, because that's just the default. The same goes for "--mixed-refs")

[Bleck]

qcomp was at 0.60 by default from a range between 0.00 and 1.00. Since is a bitrate distribution thing, I prefer it to be balanced. Other people use 0.50 too.
The idea with this thread is because I have seen reports of problems using trellis and adaptive quantization at the same time.
And crf 27 looks good for me (2xxx kbps for a 960x540p video, noticeable blur in dark gradients only).

[LoRd_MuldeR]

qcomp was at 0.60 by default from a range between 0.00 and 1.00. Since is a bitrate distribution thing, I prefer it to be balanced.

Not really an argument. The choice to make the qcomp scale 0.00 to 1.00 is completely arbitrary.

They could as well have made it from 42.0 to 666.0. So would you blindly use 354.0 then, just because it lies in the middle?

Other people use 0.50 too.

Not an argument either. A lot of people use a lot of bizarre settings ;)

Actually, nowadays x264 uses MB-Tree, unless you explicitly disabled it, and thus the qcomp setting controls the "strength" of the influence of MB-Tree.

If your tests showed that with stronger (or weaker) MB-Tree, as compared to the default value, the results looked better, then it's reasonable to adjust qcomp accordingly.

But otherwise just keep the default!

The idea with this thread is because I have seen reports of problems using trellis and adaptive quantization at the same time.

Did you actually reproduce the problem with your setup? If not, I wouldn't worry too much about "dubious" problems other people are speculating about...

If there was a known problem with using AQ and Trellis together, do you really think that the x264 developers would have made exactly that combination the default? :devil:

And crf 27 looks good for me (2xxx kbps for a 960x540p video, noticeable blur in dark gradients only).

The resulting bitrate for a specific CRF value completely depends on the individual source. But if CRF=27 satisfies your quality needs, fine.

[Bleck]

Good, I will use trellis 2 and default AQ then:

x264.exe --demuxer avs --crf 25 --bframes 16 --b-adapt 2 --direct auto --no-fast-pskip --me umh --merange 24 --partitions all --rc-lookahead 60 --ref 16 --subme 11 --trellis 2 --output "C:\Users\Developer\Desktop\video.264" %1

[x265]

Don't play around with the options if you don't know what they do.