x264 presets discussion, again

[Dark Shikari]

I didn't mean to connect it in that sense :) it could be also suggested to have --me-prepass enabled when UMH mode is selected (no point for ESA I believe?...), just as a matter of principle, since mode 7 or UMH are usually selected for quality.That would not be a good idea IMO, as --me-prepass gives considerable speed loss and not as much quality gain as UMH does over HEX, so its better to leave it up to the user.

One good idea I did have was to build in some presets to x264. They would be profile-compliant and give users a nice way of choosing options simply even if not using a GUI. Example:

--profile high --quality 85 might use something like

--bframe 16 --b-pyramid --ref 3 --me umh --me-prepass --mixed-refs --subme 7 --b-rdo --trellis 2 --bime --weightb --partitions all --8x8dct

--profile high --quality 95 might use something like

--bframe 16 --b-pyramid --ref 6 --me tes --me-prepass --mixed-refs --subme 7 --b-rdo --trellis 2 --bime --weightb --partitions all --8x8dct

--profile high --quality 100 might use something like

--bframe 16 --b-pyramid --ref 16 --me tes --me-prepass --mixed-refs --subme 7 --b-rdo --trellis 2 --bime --weightb --partitions all --8x8dct

--profile high --quality 40 might use something like

--bframe 16 --b-pyramid --me hex --subme 4 --trellis 1 --weightb --partitions i8x8,i4x4,b8x8,p8x8 --8x8dct

--profile high --quality 30 might use something like

--bframe 16 --b-pyramid --me hex --subme 3 --weightb --partitions i8x8,i4x4 --8x8dct

--profile high --quality 10 might use something like

--bframe 16 --b-pyramid --me dia --subme 2 --weightb --partitions none

All settings would be changed appropriately to fit each profile based on the --profile command used. This means you could quite literally encode, say, for an iPod, with nothing but --bitrate, --pass, --level, --profile, and --quality. Any extra commandlines would override those specified in the preset. Using the presets would be completely optional. Also note that when --profile is used, --level will automatically enforce the correct maximum bitrate given that profile.

[Adub]

I think this is a really good idea. For those people who
a.) can't use a command line very well
b.) don't know all of the numerous options that are contained in x264

it would be a very handy thing. I bet it would make things a little easier for external xml profile creation as well, like in MeGUI.

[le_canz]

I agree, it's a good idea, x264's command lines are very long sometimes.

[foxyshadis]

Wouldn't it be more in keeping with x264's philosophy to distribute it with a batch/shell script that just converts profiles into x264 options?

[RaynQuist]

If it's out of 100, people would think it's a percentage of some sort. Something arbitrary would work better, like FLAC's 0~8. It should probably be called something other than "quality", like compression-level.

[Dark Shikari]

If it's out of 100, people would think it's a percentage of some sort. Something arbitrary would work better, like FLAC's 0~8. It should probably be called something other than "quality", like compression-level.
I was thinking in the style of the VC-1 encoder, which has a similar 0-100 scale.

My philosophy would be to make 50 the best balance of speed and quality--pick all the options that maximize speed without too much quality loss, and all the options that maximize quality without too much speed loss. It wouldn't use RDO or anything of the sort.

20 or 30 would be appropriate for a first pass, and 70-80 for general high-quality use. 90-100 and 0-10 would both be considered "insane".

[LoRd_MuldeR]

x264's command lines are very long sometimes.

That's what front-ends were made for :p

People who use x264 CLI directly should be able to handle long command-lines. All others should get a nice front-end...

[fields_g]

I was thinking in the style of the VC-1 encoder, which has a similar 0-100 scale.

My philosophy would be to make 50 the best balance of speed and quality--pick all the options that maximize speed without too much quality loss, and all the options that maximize quality without too much speed loss. It wouldn't use RDO or anything of the sort.

20 or 30 would be appropriate for a first pass, and 70-80 for general high-quality use. 90-100 and 0-10 would both be considered "insane".

One thing to think about is that x264 encoders don't only balance speed and quality, but also compression (file size). Any of the two factors will dictate the third, but which factor do you want to be the third one? This 3D relationship, I feel, makes command line profile presets a little difficult to target.

Also with the ipod example, there are many hardware players that do not stop support at h.264 level/profile borders and support for other features (cabac, b-rdo, pyramid, etc.) is widely varied. I worry that the demand for presets for each new hardware platform to be built into x264 might become overwhelming/ridiculous.

I feel we can leave these problems to GUIs and their developers.

[Dark Shikari]

One thing to think about is that x264 encoders don't only balance speed and quality, but also compression (file size). Any of the two factors will dictate the third, but which factor do you want to be the third one? This 3D relationship, I feel, makes command line profile presets a little difficult to target.
Compression is equivalent to quality, because lower file size = more quality can be fit in the same file size. Its not 3 dimensional.

Also with the ipod example, there are many hardware players that do not stop support at h.264 level/profile borders and support for other features (cabac, b-rdo, pyramid, etc.) is widely varied. I worry that the demand for presets for each new hardware platform to be built into x264 might become overwhelming/ridiculous.
That is irrelevant--x264 should abide by the standard, and the standard only specifies 3 profiles that affect us: Baseline, Main, and High. If someone wants to add an extra feature to a profile for some strange hardware player that doesn't abide by the official standard for some ungodly reason, they can add the appropriate commandline.

I feel we can leave these problems to GUIs and their developers.Perhaps, but many other commandline encoders have similar features to the one I suggested. You do have a point though; it might not really be in x264's philosophy to have such an option. On the other hand, it would be a huge convenience, and would probably make x264 more popular independent of any specific GUI.

[Sagittaire]

That would not be a good idea IMO, as --me-prepass gives considerable speed loss and not as much quality gain as UMH does over HEX, so its better to leave it up to the user.

One good idea I did have was to build in some presets to x264. They would be profile-compliant and give users a nice way of choosing options simply even if not using a GUI. Example:

--profile high --quality 85 might use something like

--bframe 16 --b-pyramid --ref 3 --me umh --me-prepass --mixed-refs --subme 7 --b-rdo --trellis 2 --bime --weightb --partitions all --8x8dct

--profile high --quality 95 might use something like

--bframe 16 --b-pyramid --ref 6 --me tes --me-prepass --mixed-refs --subme 7 --b-rdo --trellis 2 --bime --weightb --partitions all --8x8dct

--profile high --quality 100 might use something like

--bframe 16 --b-pyramid --ref 16 --me tes --me-prepass --mixed-refs --subme 7 --b-rdo --trellis 2 --bime --weightb --partitions all --8x8dct

--profile high --quality 40 might use something like

--bframe 16 --b-pyramid --me hex --subme 4 --trellis 1 --weightb --partitions i8x8,i4x4,b8x8,p8x8 --8x8dct

--profile high --quality 30 might use something like

--bframe 16 --b-pyramid --me hex --subme 3 --weightb --partitions i8x8,i4x4 --8x8dct

--profile high --quality 10 might use something like

--bframe 16 --b-pyramid --me dia --subme 2 --weightb --partitions none

All settings would be changed appropriately to fit each profile based on the --profile command used. This means you could quite literally encode, say, for an iPod, with nothing but --bitrate, --pass, --level, --profile, and --quality. Any extra commandlines would override those specified in the preset. Using the presets would be completely optional. Also note that when --profile is used, --level will automatically enforce the correct maximum bitrate given that profile.

You can make that only for ME/RDO search like subme, me, brdo ... etc

Moreover use brdo is for example very bad for grain preservation. For me this quality profil is simply useless.

[Sagekilla]

You can make that only for ME/RDO search like subme, me, brdo ... etc

Moreover use brdo is for example very bad for grain preservation. For me this quality profil is simply useless.

That's true, but honestly speaking if you're aiming for grain retention why would you be using preset profiles? You'd be better off using settings manually since you'd get the greatest flexibility that way. What Dark Shikari is suggesting sounds more like something someone inexperienced with x264 would be using. Plus, if there was a real need for grain retention you could always add in a --grain switch or something of the sort to disable the appropriate settings at whatever level you use to help retain grain.

[fields_g]

Compression is equivalent to quality, because lower file size = more quality can be fit in the same file size. Its not 3 dimensional.
ok... AT A GIVEN BITRATE altering compression is equivalent to altering picture quality.

You need 3 values to describe how a frame was encoded. For example, one could be 50k, 0.850 ssim(insert favorite objective or subjective measure), and take 0.1 seconds to encode. I cannot alter "improve" one of these without taking away from one of the other two or both (unless you are using some really crappy settings). The person encoding has a choice. The entire x264 command line dictates this 3D relationship. Your proposed quality scale would need to be paired with bitrate.

[Dark Shikari]

ok... AT A GIVEN BITRATE altering compression is equivalent to altering picture quality.

You need 3 values to describe how a frame was encoded. For example, one could be 50k, 0.850 ssim(insert favorite objective or subjective measure), and take 0.1 seconds to encode. I cannot alter "improve" one of these without taking away from one of the other two or both (unless you are using some really crappy settings). The person encoding has a choice. The entire x264 command line dictates this 3D relationship. Your proposed quality scale would need to be paired with bitrate.--quality would only control quality options, it doesn't affect ratecontrol. The user would still have to specify --crf, --qp, or --bitrate.

[Dark Shikari]

Good idea to split off the thread.

[check]

I never like options that override other options in CLI programs. The behavior is never intuitive. What if you specify --me umh --preset fast? What about --preset fast --me umh? It makes a lot more sense to implement profiles as a wrapper, a la MeGUI and others.

As to the whole '3d relationship' thing, isn't it reasonable to describe the relationship as 'time' vs. 'efficiency', where efficiency = quality vs. compression? Then you somewhat simplify the relationship, which would be useful for presets.

[Dark Shikari]

After a whole lot of testing, here's the presets I came up with for optimal quality at a particular speed:

Q100 --bframe 16 --me tes --merange 24 --subme 7 --no-fast-pskip --partitions all --8x8dct --bime --ref 16 --trellis 2 --mixed-refs --weightb --me-prepass --b-rdo
Q95 --bframe 16 --me esa --merange 16 --subme 7 --no-fast-pskip --partitions all --8x8dct --bime --ref 10 --trellis 2 --mixed-refs --weightb --me-prepass --b-rdo
Q90 --bframe 16 --me umh --merange 24 --subme 7 --no-fast-pskip --partitions all --8x8dct --bime --ref 8 --trellis 2 --mixed-refs --weightb --me-prepass --b-rdo
Q85 --bframe 16 --me umh --merange 16 --subme 7 --no-fast-pskip --partitions all --8x8dct --bime --ref 6 --trellis 2 --mixed-refs --weightb --me-prepass --b-rdo
Q80 --bframe 16 --me umh --merange 16 --subme 7 --partitions all --8x8dct --bime --ref 6 --trellis 1 --mixed-refs --weightb --me-prepass --b-rdo
Q75 --bframe 16 --me umh --merange 12 --subme 7 --partitions all --8x8dct --bime --ref 5 --trellis 1 --mixed-refs --weightb --b-rdo
Q70 --bframe 16 --me umh --merange 12 --subme 6 --partitions all --8x8dct --bime --ref 5 --trellis 1 --mixed-refs --weightb
Q65 --bframe 16 --me umh --merange 8 --subme 5 --partitions i8x8,b8x8,p8x8,i4x4 --8x8dct --bime --ref 5 --trellis 1 --mixed-refs --weightb
Q60 --bframe 16 --me hex --subme 4 --partitions i8x8,b8x8,p8x8,i4x4 --8x8dct --bime --ref 5 --trellis 1 --mixed-refs --weightb
Q55 --bframe 16 --me hex --subme 4 --partitions i8x8,b8x8,p8x8,i4x4 --8x8dct --bime --ref 4 --mixed-refs --weightb
Q50 --bframe 16 --me hex --subme 3 --partitions i8x8,b8x8,p8x8 --8x8dct --bime --ref 4 --mixed-refs --weightb
Q45 --bframe 16 --me hex --subme 3 --partitions i8x8,b8x8,p8x8 --8x8dct --ref 3 --mixed-refs --weightb
Q40 --bframe 16 --me hex --subme 3 --partitions i8x8,b8x8,p8x8 --8x8dct --ref 2 --mixed-refs --weightb
Q35 --bframe 16 --me dia --subme 3 --partitions i8x8,b8x8,p8x8 --8x8dct --weightb
Q30 --bframe 16 --me hex --subme 3 --partitions i8x8,b8x8 --8x8dct --weightb
Q25 --bframe 16 --me dia --subme 3 --partitions i8x8,b8x8 --8x8dct --weightb
Q20 --bframe 16 --me hex --subme 3 --partitions b8x8 --weightb
Q15 --bframe 16 --me hex --subme 3 --partitions none --weightb
Q10 --bframe 16 --me hex --subme 2 --partitions none --weightb
Q5 --bframe 16 --me dia --merange 12 --subme 2 --partitions none
Q0 --bframe 16 --me dia --merange 8 --subme 1 --partitions none

[kikker]

There are many hardware devices that require a pretty stringent combination of h.264 features. And, there a now quite a few profiles geared for each class of hardware, that would make these profiles a bit meaningless. Secondly, who is the target audience for the creation of these profiles? Do they provide "guidelines" for the profile creators, the GUI developers, conventient shorthand for those who encode directly with the command-line? I would say, that only the latter group might be interested, and even then, there would be a whole lot of overriding going on, I can well imagine. But, simply agreeing on a set of presets without coding them into the encoder would certainly help relative noobs like myself gain a better understanding of some of the features.

[akupenguin]

Interesting that you enable b8x8 before p8x8. Perhaps it's because bskip is so aggressive that not many mbs remain to go though mode decision?

[Dark Shikari]

Interesting that you enable b8x8 before p8x8. Perhaps it's because bskip is so aggressive that not many mbs remain to go though mode decision?
From my testing:

Disabling b8x8: +0.87% speed, -3.5% quality
Disabling p8x8: +10.5% speed, -7.6% quality

[Eastermeyer]

Where can i get a Dark Shikari Build (680) ?

[Dark Shikari]

Where can i get a Dark Shikari Build (680) ?You can get one when I get off my ass and fix up the patches, and then convince Akupenguin to put the subme7 and meprepass patches into the SVN :p

[Sagekilla]

You can get one when I get off my ass and fix up the patches, and then convince Akupenguin to put the subme7 and meprepass patches into the SVN :p

I noticed up top for the highest quality settings, you have a ME mode called "tes." What's that? In the past few weeks that I've been following the patchwork floating around the forums, I've never noticed any ME called tes.

[Dark Shikari]

I noticed up top for the highest quality settings, you have a ME mode called "tes." What's that? In the past few weeks that I've been following the patchwork floating around the forums, I've never noticed any ME called tes.It just means --fpel-cmp satd + --me esa, aka Transformed Exhaustive Search. Aku and I decided it would be better as a new ME mode, as all the other ME modes are not worthwhile with --fpel-cmp satd.

[akupenguin]

Why are Q85 and Q90 identical?

[Dark Shikari]

Why are Q85 and Q90 identical?A mistake by me when spacing out the top few values; I made one fewer than I needed so one ended up duplicated. It'll be fixed in a few minutes.

[Eastermeyer]

So it's not out yet. Thanks for the information.
Hope i can get this soon.

[Adub]

I think it is more of a build with Dark Shikari's patch. Cef might be building one. Not sure. Check the patches section on x264.nl's website.

[Eastermeyer]

Yeah, i found that http://mirror05.x264.nl/Cef/force.php?file=./x264_x86_dark.7z
, but it is an old version (676) and an option --me tes isn't supported.

[Dark Shikari]

Yeah, i found that http://mirror05.x264.nl/Cef/force.php?file=./x264_x86_dark.7z
, but it is an old version (676) and an option --me tes isn't supported.--me tes has never existed; its just a new name for --fpel-cmp satd and --me esa, which if the Hadamard patch is put into SVN, will replace the --fpel-cmp commandline.