Higher parameters give bad results - why?

[Solon8]

Hi everyone,

As I got a new and faster computer, I decided to choose higher values for some parameters in x264. The encoding speed was still way lower than on my old computer, so I was thrilled but after checking the final file, I realized that the results were not so great for some videos and I had to start over with a higher bitrate. But since it happened often, I decided to see if the bad results were the same with my old preset. Test results: I have a better rendering with lower values... I don't understand, I thought, now my computer allows me to, that increasing some parameters (such as subme, merange, rc_lookahead) could only make things better.

The "new" parameters were:
cabac=1 / ref=5 / deblock=1:-1:-1 / analyse=0x3:0x113 / me=umh / subme=9 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=20 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-4 / threads=12 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=0 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=240 / keyint_min=23 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=2pass / mbtree=1 / bitrate=941 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / ip_ratio=1.40 / aq=2:1.00

The "old" ones, that I use now with cleaner results are:
cabac=1 / ref=5 / deblock=1:-1:-1 / analyse=0x3:0x113 / me=umh / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-4 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=0 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=240 / keyint_min=23 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=2pass / mbtree=1 / bitrate=900 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / ip_ratio=1.40 / aq=1:1.00

Does anyone know why I have better results (on different videos, of different types) with lower values. Am I using one or two parameters wrong that make things worse? Maybe while using a higher subme, I should change some of those parameters? (trellis? me? merange?)

I don't know much about all those, I tried to read a few wiki but I thought you guys might help. Thanks!

[LoRd_MuldeR]

There are (at least) two categories of parameters:

1. The first category contains parameters where it's all about finding the "sweet spot", so neither setting the parameter too high nor setting it too low will give good result! The "Psy" parameters generally fall into that category.

2. The second category contains "speed vs. quality" parameters, i.e. parameters that control the trade-off between encoding speed and visual quality (at a given bitrate). Here, as a rule of thumb, you can say "the higher the better (but slower)".


Anyway, even within the second category, just cranking up everything to the maximum isn't always a good idea. Often there is a point where "going even higher" only gives a very small improvement (if at all), but causes a significant speed loss!

Furthermore, with some options (e.g. ME-Range), using "extreme" values probably doesn't help at all and can even be self-defeating in some cases. I would suggest you simply use "--preset slower" or even "--preset veryslow" and that's it :)

And if "--preset slower" or "--preset veryslow" really produces worse quality than, e.g., "--preset medium", at the same average bitrate and from the identical source, then you should post an example!

(BTW: How exactly did you determine that the "slower" settings resulted in "worse" quality ???)

[Solon8]

Hi, thanks for your reply.

I agree with you, those changes won't be of any significance (I also noticed the birate is always much more important than anything) but since I got this new, fast computer I wanted to try something higher and the encoding time is still way lower than on my previous computer.

I just found it strange that, for a same video, subme 7 gives better results than subme 9. I really thought the higher, the better for that one. Can some parameters ruin others?

But was I wrong to choose subme 9, merange 20, rc_lookahead between 40 and 60? It doesn't seem too much, I've seen this on many other encoding.

To answer your question, I noticed some, I don't know, bluriness or something or at least a visual aspect I usually have with lower bitrates (too low bitrates). That's what was weird for me, that I had to choose higher bitrates to "solve" the problem, while with my old, faster settings, the bitrate I was always using were perfectly fine.

I guess, as you say, I'll use premade slower presets from now on, maybe something was wrong in my preset.

[LoRd_MuldeR]

I just found it strange that, for a same video, subme 7 gives better results than subme 9. I really thought the higher, the better for that one.

Well, a higher value here means that the encoder is likely to find a more "optimal" decision - simply because more possibilities will be evaluated.

But you need to be aware that "more optimal" here means the following: more optimal with respect to whatever measure the encoder uses to determine the "goodness" of the particular decision.

Usually we hope that a "better" decision in this regard also means that the result will be more visually pleasing to the human viewer. That's the goal. But there's no absolute guarantee for that!

(The measure that perfectly predicts human visual perception is still to be developed ;))


[EDIT]

Here is an interesting article that explain why most H.264 encoders suck, because they optimize (too much) for a particular measure, namely PSNR/MSE:
http://x264dev.multimedia.cx/archives/164

[Groucho2004]

The change that makes a difference is this:
"aq=1:1.00" -> "aq=2:1.00"

Changing the AQ Mode can have a big impact. I suggest you stick to the presets if you don't know what the settings do.

Also, have a look here (http://mewiki.project357.com/wiki/X264_Settings).

[LoRd_MuldeR]

The change that makes a difference is this:
"aq=1:1.00" -> "aq=2:1.00"

@Solon8:
Here you need to realize that setting "--aq-mode" to 2 chooses a different AQ algorithm, but that other algorithm is not necessarily better than mode 1 (aka "default" AQ mode).

There's probably a reason why mode 1 is still the default. The x264 devs probably would have made mode 2 the default, if it consistently performed better than mode 1 :)

[Solon8]

Interesting article indeed, LoRd_MuldeR but I can't say I understood it all, which makes me think even more that I should stick to presets.

Honestly, I don't remember why I chose aq-mode 2. I have two links I used when choosing my settings: the link Groucho2004 just sent and this one (http://en.wikibooks.org/wiki/MeGUI/x264_Settings#aq-mode), which sets aq-mode 2 as default, not sure why.

I will try slower premade presets, see if "it" still happens or not.

:thanks: