H.264 Profiles and Levels discussion [Archive]

LoRd_MuldeR

10th October 2014, 22:28

Hi, I was wondering how your program chooses the correct Profile when setting it to Auto? Does it always choose the correct profile?

It doesn't! If you select "Auto", then the "--profile" option isn't used at all, which means no specific profile is being enforced, which means x264 will set the proper Profile automatically.

Also is there a way I can choose the exact level such as High 3.2 instead of just setting "High" and then the program chooses which level to set?

You seem to have a rather strange understanding of what Profiles and Levels do. And I think we had that discussion before ;)

Profiles (http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Profiles) are restrictions of what features of H.264 you are allowed to use. By default, if "--profile" is not used, x264 will pick the Profile that covers all features you are using in your stream (but not a higher one, since that would be pointless). Consequently, you can enforce a lower Profile than what would be used by default, which will simply turn off those features that are not allowed in the enforced Profile. Enforcing a higher Profile than what would be used by default doesn't make any sense though! What feature should x264 turn on to make your file require the higher Profile? Should it pick a random one? Instead, if you want your stream to be "High" Profile, then simply use at least one "High" Profile feature and that's it!

Levels (http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Levels) are quite similar to Profiles, in the sense that they also are restrictions. Though Levels restrict other things, such as the maximum resolution, the maximum frame rate, the maximum number of reference frames, and so on. x264 will set the Level that is suitable for your stream (and not a higher one, since that would be pointless). And what Level your stream needs is defined by your input file! You can use "--level" to overwrite the Level. However, if you enforce a Level that is lower than what x264 would have set by default, x264 cannot do much about this, except for showing a warning message that your stream does not actually comply to the Level you have set. And enforcing a Level that is higher than what x264 would have set by default only restricts the compatibility of your stream for zero benefit! To make a long story short: Setting "--profile" can make sense in specific cases, but setting "--level" manually is almost always a bad idea. Simply let x264 do its job...

VideoFanatic

10th October 2014, 23:17

I understand all that but I'm trying to make a compliant Bluray. For example I'm trying to convert a 1080i PAL HDTV h264 video to 720p 50fps. If you look at the profiles table here: http://forum.doom9.org/showthread.php?t=154533 (under 1.2.1 Primary Video Rules) it shows 720p must have a 4.0 Level. Yet I chose the auto setting and it came out as High Level 3.2.
I was using these settings:

--bluray-compat --vbv-maxrate 15000 --vbv-bufsize 15000 --keyint 250 --slices 1 --sar 1:1

So I'm just wondering why it doesn't come out as 4.0 because from what I've seen in the table, it should come out as 4.0?

LoRd_MuldeR

10th October 2014, 23:35

First of all, when it comes to Levels, a decoder/player that supports Level X actually supports Levels up to X. Feeding the player with a stream that has a lower Level than the player's max. supported Level should never be an issue.

Or in other words: A video stream that complies to Level X also complies to any Level higher than X, tough it may not comply to Levels below X.

Since the BluRay standard is a subset of the H.264 standard, i.e. it only supports certain H.264 Profiles and H.264 Levels but there are even more restrictions on top of that, I don't know if supplying a stream that also complies to lower Levels than 4.0 would be acceptable. Anyway, if you look at the Levels Table (http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Levels), you'll notice that Level 3.2 supports 1280x720 video with up to 60 fps and 5 ref frames, so there's simply no need for Level 4.0 if you are dealing with 720p video at 50 fps.

To give a car analogy: You are demanding a speed limit of 120 km/h instead of 100 km/h despite your car's maximum speed is only 80 km/h anyway ;)

See also:
http://www.x264bluray.com/home/720p-encoding

r0lZ

11th October 2014, 09:48

I'm not sure I understand that profile and level concepts correctly. I did some tests with different presets (CRF 23, profile auto and level not defined), of course always with the same input file, and obviously the profile and levels depend of the input file, that's right, but also of the preset. For example, I got the Baseline profile when I have encoded the video with the Ultrafast preset, and profile high level 5.1 when the same video has been encoded with the Placebo preset, without any other change. Since my TV refuses to display level 5.0 or higher, I NEED to specify --level 4.1 if I use a slow preset.

Therefore, if I can agree that the characteristics of the input file are important to determine the profile and level used by default by x264, I notice that the preset is at least as important, and I must disagree when you say that "setting --level manually is almost always a bad idea". IMO, setting level to 4.1 (or max 4.2) is essential if you need a good compatibility with most hardware players, and you want to encode with a slow preset. (I wonder if it make sense to use a very slow preset like Placebo and in the same time limit the encoder with --profile high --level 4.1, but it's another question.)

LoRd_MuldeR

11th October 2014, 13:48

Again, the Profile defines which features of H.264 you are allowed to use. See this (http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Profiles) table for details or look into the H.264 specs (Annex A.2). Which Profile your stream comes out shouldn't depend on the input file at all. Instead it depends on what H.264 features you enable (or not enable). Of course features that influence the required Profile can be enabled/disabled either directly, by the corresponding switch, or indirectly, by the "--preset" switch.

Consequently, your stream will come out at the lowest possible Profile that covers all the features you were using. It cannot come out at an even lower Profile, because that's obviously not allowed by the specs. And it also doesn't come out at an even higher Profile, because that would be totally pointless - your stream doesn't need the even higher Profile, so why should an even higher Profile be indicated? It would only limit the compatibility of the stream - for exactly zero benefit!

It's simply the wrong idea to "want" a certain Profile. Instead, the target Profile is given by your intended playback device! If, for example, you playback device only supports "Main" Profile, you need to restrict your settings to "Main" Profile features. And, if your playback device supports "High" Profile, you can use "High" Profile features. Last but not least, if you are allowed to use "High" Profile features and if you actually do enable at least one "High" Profile feature, your stream will come out as "High" Profile automatically. The "--profile" switch is not intended to force a higher Profile; it is intended to enforce at most the specified Profile. Therefore, "--profile X" will turn off all features not allowed in Profile X. And that's it.

The Level restricts other things, like the max. resolution, the max. frame rate, the max. number of reference frames, the max. bitrate and so on. So what Level your stream requires partly depends on the input file (resolution, frame rate, etc) and also partly on your encoder settings (number of reference frames, bitrate, etc). By default, if you do not use the "--level" switch, x264 will set the lowest possible Level that your stream complies to. And that's the right thing to do!

The reason why I said that using "--level" is almost always a bad idea is because x264 will already set the "correct" Level, if you do not use that switch. Let's assume x264 would set Level X by default. Why would you want to enforce a lower Level than X, i.e. a Level that your stream actually doesn't comply to? It would result in a corrupted stream, because the stream actually requires Level X, but it has been tagged as some Level lower than X. Similarly, why would you want to set a Level higher than X? Your stream complies to Level X, so tagging it as some Level higher than X is pointless. You would restrict the compatibility of your stream for zero benefit! The only reason to use "--level" is when x264 somehow got the Level wrong.

detmek

11th October 2014, 14:19

LoRd_MuldeR

11th October 2014, 14:34

Well, I know one use for --level. If you encode FHD video with preset placebo it will end up as level 5.1. But, if you use --level 4.1 --preset placebo video will end up as level 4.1 because --level 4.1 will reduce number of reference frames from 16 to 4.

No, that's wrong. If your stream would normally come out as Level 5.1, but you enforce Level 4.1 by adding "--level 4.1" (while keeping all other settings the same), you stream's Level requirements do not change at all. Only difference is that your stream will now incorrectly be tagged as Level 4.1, while it actually doesn't comply to that Level - it still requires Level 5.1.

Using the "--level" switch does not reduce the number of reference frames. Instead, x264 will print out a warning, telling you that the selected number of reference frames violates the selected Level:
> x264_8bit_x64.exe --crf 18.0 --preset slower --ref 16

ffms [info]: 1920x1080p 0:1 @ 24000/1001 fps (vfr)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64 SlowShuffle
x264 [info]: profile High, level 5.1> x264_8bit_x64.exe --crf 18.0 --preset slower --ref 16 --level 4.2

ffms [info]: 1920x1080p 0:1 @ 24000/1001 fps (vfr)
x264 [warning]: DPB size (16 frames, 130560 mbs) > level limit (4 frames, 34816 mbs)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64 SlowShuffle
x264 [info]: profile High, level 4.2

If you want fewer reference frames, simply use the "--ref" switch! And, if this actually reduced your stream's Level requirements, x264 will set the new (lower) Level automatically!

r0lZ

11th October 2014, 14:34

There is no need to explain that again. Re-read what I have written. I understand perfectly that the --profile and --level options are the UPPER LIMITS. And I don't "want" high@L4.1. I haven't said that. I just want to limit x264 so that it creates a stream that my TV can play (and for the precise example of my TV, the limit is high@L4.2).

What I mean is that if you use a very slow PRESET (without limiting x264 to a certain profile or level), then the profile and/or level may be higher than what you expect, and therefore it might be NECESSARY to use the --profile and --level options. In the past, I used your GUI without specifying the level, but one time I did an encode and my TV has rejected it. That was because it was a very short clip, and for that reason, I've used for the first time a very slow preset. After verification, I came to the conclusion that the profile and level really used by x264 depend of the preset. That's what I have explained above, and it's a verified fact. Since then, I have added --level 4.1 in the user options, and my TV has never rejected any encoding any more. I understand perfectly that my TV can accepts lower presets or levels, and that x264 can produce them, even if I have specified high@L4.1.

What I mean is that it is incorrect to consider the level option as not important. In many case, it is an absolute necessity to specify it. I don't want to create videos that my TV rejects, and therefore I need to specify the --level option (at least when I use a slow preset). Again, it's a fact. (Of course, I could also not specify --level, but specify a lot of other options, that together have globally the same effect than using --level 4.1, but that's not the point.)

It's why I don't understand why the level option is not present in your GUI, and you have to type it explicitly. It's a very important setting that should be set to 4.1 by default IMO, because (almost) all recent players can decode that level, but currently very few hardware players accept level 5.0 or higher. IMO, NOT specifying the level IS the bad idea, because you may end up with files that are rejected by many players.

LoRd_MuldeR

11th October 2014, 14:49

What I mean is that if you use a very slow PRESET (without limiting x264 to a certain profile or level), then the profile and/or level may be higher than what you expect, and therefore it might be NECESSARY to use the --profile and --level options.

What do I expect? I expect that, if I neither use "--profile" nor "--level", then x264 will set the "correct" Profile and Level for me. That is: the lowest possible Profile that my stream complies to as well as the lowest possible Level that my stream complies - given the current input file and the current encoder settings. Consequently, manually enforcing an even higher Profile or an even higher Level would make no sense. Enforcing a lower Profile can make sense, e.g. if I need to stick within "Main" Profile, because my playback device can only do "Main" Profile but not "High" Profile. Enforcing a lower Level, on the other hand, is almost always a very bad idea, as explained and demonstrated in the previous post (https://forum.doom9.org/showpost.php?p=1696566&postcount=1395).

Can I request a small feature please. At the moment when you modify a profile it gives you the option to select a profile to overwrite. But I can't always remember the name of the profile I was editing and want to overwrite because it has a long name.

You can't modify Profiles, because they are predefined by the H.264 standard. All H.264 Profiles that x264 recognizes should be selectable from the "Profile" combobox.

r0lZ

11th October 2014, 15:31

What do I expect? I expect that, if I neither use "--profile" nor "--level", then x264 will set the "correct" Profile and Level for me.
I agree. But that doesn't mean that you don't need to specify the level when your target hardware is not compatible with the highest levels.

That is: the lowest possible Profile that my stream complies to as well as the lowest possible Level that my stream complies - given the current input file and the current encoder settings.
Here, I disagree, or I doesn't understand what you mean. If it's really the lowest profile that is selected, all videos should be encoded in the baseline profile, the lowest one. (It is compatible with the clip I have encoded, since it's the profile that x264 uses by default if I select the ultrafast preset.) IMO, the encoder tries to use the highest profile and level, given the input file and the preset. It's what I have verified: when you give it more time to compute a higher profile or level, it does it (of course, as long as it's compatible with your file).

Enforcing a lower Level, on the other hand, is almost always a very bad idea, as explained and demonstrated in the previous post (https://forum.doom9.org/showpost.php?p=1696566&postcount=1395).

If your stream would normally come out as Level 5.1, but you enforce Level 4.1 by adding "--level 4.1" (while keeping all other settings the same), you stream's Level requirements do not change at all. Only difference is that your stream will now incorrectly be tagged as Level 4.1, while it actually doesn't comply to that Level - it still requires Level 5.1.
Where have you learned that? In all sites about x264 I have visited, the level option is always presented as the upper limit for the encoder, not just as a way to force a (possibly wrong) information in the header. Also, my TV supports only level 4.2. If I encode without --level 4.2 at preset placebo, x264 uses level 5.1, and writes that info in the header. My TV rejects it. OK, that might be because it has read the info in the header. But how could you explain that when I encode the same clip with exactly the same parameters except that I have added --level 4.1, the same TV plays it perfectly? My TV rejects level 5.1 because it can't play it, but plays perfectly my file. Therefore, my file has been encoded at level 4.1, and the --level option has really been used to limit the encoder during the encoding process. Otherwise, I should at least see some glitches from time to time, but the result is perfect. Don't tell me that my TV can decode level 5.1 but rejects it, just to annoy the peoples.

However, I agree that specifying --ref 16 AND the --level 4.1 at the same time is incompatible and will trigger warnings. Of course, if you limit x264 to a certain level AND you force it to use some characteristics from a higher level, it will not be able to do what you want without violating one of the parameters. And in that case, it may write level 4.1 in the header, even if it's not true. In my tests, I specify ONLY the level, without other incompatible options, and in that case, I'm sure x264 produces really a stream compatible with level 4.1.

LoRd_MuldeR

11th October 2014, 15:48

Here, I disagree, or I doesn't understand what you mean. If it's really the lowest profile that is selected, all videos should be encoded in the baseline profile, the lowest one.

If you don't use "--profile" explicitly, then x264 will set the lowest possible Profile that your stream complies to - given the current encoder settings. That's what I said ;)

Or in other words: If you use lossless mode, it will set "High 4:4:4" Profile. If you do not use lossless mode, but you have any "High" Profile features enabled, it will set "High" Profile. If you have no "High" Profile features enabled, but some "Main" Profile features are enabled, it will set "Main Profile". Otherwise, if only "Baseline" Profile features are used, it will set "Baseline" Profile.

The "--profile X" option can be used to enforce a lower profile, because it will simply disable all the features that are not allowed in Profile X (if any of those were enabled before).

Where have you learned that? In all sites about x264 I have visited, the level option is always presented as the upper limit for the encoder, not just as a way to force a (possibly wrong) information in the header. Also, my TV supports only level 4.2. If I encode without --level 4.2 at preset placebo, x264 uses level 5.1, and writes that info in the header. My TV rejects it. OK, that might be because it has read the info in the header. But how could you explain that when I encode the same clip with exactly the same parameters except that I have added --level 4.1, the same TV plays it perfectly? My TV rejects level 5.1 because it can't play it, but plays perfectly my file. Therefore, my file has been encoded at level 4.1, and the --level option has really been used to limit the encoder during the encoding process. Otherwise, I should at least see some glitches from time to time, but the result is perfect. Don't tell me that my TV can decode level 5.1 but rejects it, just to annoy the peoples.

As said before, if you use the "--level" switch, it will simply overwrite the Level that x264 would have set normally. And the Level that x264 would have set normally is the lowest possible Level that your stream complies to - given the current input file and the current encoder settings. Consequently, if you set an even lower Level, you stream will be tagged with that lower Level, yes. But that does not make your stream actually comply to that even lower Level! There is nothing in x264 that would "fix" your encoder settings (or your input file) to actually make the stream conform to the Level that you have set via "--level" switch! You can check this in the x264 code, if you want to...

And you can test this easily: Set a large number of reference frames via "--ref" switch and force a Level that doesn't actually allow that many reference frames (due to DPB size limit) via "--level" switch. Will x264 lower the number of reference frames for you, to make your stream actually comply to the enforced Level? No it doesn't! It only prints a warning message that your stream violates the Level! Does it still still set the enforced Level, despite your stream is violating the Level? Yes.

See also example here:
https://forum.doom9.org/showpost.php?p=1696566&postcount=1395

r0lZ

11th October 2014, 16:25

Consider this example:
I encode my clip without any parameter, except --preset ultrafast. The encoder uses the baseline profile. (I don't think the level is relevant for the baseline profile, but I may be wrong.)
Then, I encode the same clip with --preset placebo. The encoder uses the high profile and level 5.1.
That means that my clip complies to all profiles between baseline and high, and to all levels currently implemented (up to 5.1).
When it has enough time, x264 selects the "best" profile and level automatically. For me, and obviously for x264, "best" is synonym with "highest" and not "lowest". Am I wrong?

Your example with the --ref and --level is perfect to prove that it is possible to specify incompatible options. I have agreed with that in the last paragraph of my previous post.

Now, if you want to encode the clip of the example above and be compatible with my TV WITHOUT SPECIFYING FANCY OPTIONS, you have 3 possibilities.

1. You can use a fast preset, but the result is unpredictable, and it is a pity to waste disc space.

2. You can specify --level 4.1, and everything is perfect, regardless of the preset you use (but as long as you don't specify incompatible options at the same time).

3. You can omit --level, but specify a bunch of other options so that in practice, you obtain also level 4.1. Since many options are somewhat obscure, and you can always make a mistake, my preferred solution is, by far, point 2.

It's why IMO it is a pity to not have that level option in your GUI. Most users do not understand the complex options of x264, and should use only the main options: the mode, quantizer or bitrate and the preset, profile and level. More knowledgeable peoples may prefer to specify a lot of more specific options, but they should be smart enough to disable the profile and level options if they are incompatible.

LoRd_MuldeR

11th October 2014, 16:47

I encode my clip without any parameter, except --preset ultrafast. The encoder uses the baseline profile. (I don't think the level is relevant for the baseline profile, but I may be wrong.)

Profiles and Levels are orthogonal. The Level is as relevant (or irrelevant) with "Basline" Profile as with any other Profile.

Then, I encode the same clip with --preset placebo. The encoder uses the high profile and level 5.1.
That means that my clip complies to all profiles between baseline and high, and to all levels currently implemented (up to 5.1).
When it has enough time, x264 selects the "best" profile and level automatically. For me, and obviously for x264, "best" is synonym with "highest" and not "lowest". Am I wrong?

Totally wrong!

For the last time: The Profile specifies witch features of H.264 you are allowed to use (or not). And x264 will select the lowest possible Profile that your stream compiles to. Always. No exceptions :rolleyes:

So if you enable any "High" Profile features (e.g. "8x8 vs. 4x4 Transform Adaptivity"), as is the case with most of the slower presets (including "Placebo" preset), then your stream requires "High" Profile and that is exactly what x264 will set!

If, on the other hand, you only use "Baseline" Profile features, as is the case with "ultrafast" preset (since it disables pretty much everything that can be disabled), you obviously can get away with "Baseline" Profile. And that will be set.

Now, if you want to encode the clip of the example above and be compatible with my TV WITHOUT SPECIFYING FANCY OPTIONS, you have 3 possibilities.

1. You can use a fast preset, but the result is unpredictable, and it is a pity to waste disc space.

2. You can specify --level 4.1, and everything is perfect, regardless of the preset you use (but as long as you don't specify incompatible options at the same time).

3. You can omit --level, but specify a bunch of other options so that in practice, you obtain also level 4.1. Since many options are somewhat obscure, and you can always make a mistake, my preferred solution is, by far, point 2.

Since x264 will always set the "correct" Profile (and also the proper Level, as long as you don't use"--level") for you, there is usually no need to use either "--profile" or "--level".

Only reason the use "--profile X" is when you explicitly want to apply the restrictions of Profile X to your x264 settings, because you know that the highest Profile supported by your playback device is X.

However, the "--level" option can not be used in that way. It just overwrites the Level that your stream will be tagged with - whether your stream actually conforms to that Level or not :eek:

sneaker_ger

11th October 2014, 16:51

Using the "--level" switch does not reduce the number of reference frames.
And you can test this easily: Set a large number of reference frames via "--ref" switch and force a Level that doesn't actually allow that many reference frames (due to DPB size limit) via "--level" switch. Will x264 lower the number of reference frames for you, to make your stream actually comply to the enforced Level? No it doesn't!
It will if you do not override it with --ref. That's probably what r0lZ is seeing. But it doesn't enforce vbv so you really can end up breaking the limits. Most people don't notice because they encode with low average bitrates seldomly leading to problematic spikes and because devices don't straight out reject such streams - unlike with streams using too big DPB or too high level flag.

Using --level without --ref is handy because you don't have to calculate yourself, x264 does all the work.

r0lZ

11th October 2014, 17:13

So if you enable any "High" Profile features (e.g. "8x8 vs. 4x4 Transform Adaptivity"), as is the case with most of the slower presets (including "Placebo" preset), then your stream requires "High" Profile and that is exactly what x264 will set!

If, on the other hand, you only use "Baseline" Profile features, as is the case with "ultrafast" preset (since it disables pretty much everything that can be disabled), you obviously can get away with "Baseline" Profile. And that will be set.
OK, I understand now that the preset enforces some specifications. Until now, I thought that its only purpose was to allow more or less time to the encoder. Of course, with less time, the encoder cannot use the most sophisticated compression algorithms, and use the simplest ones, corresponding for example to baseline for ultrafast.

In my understanding, that means that if it has enough time to compute, say, profile high, it does it. I did not know that it is forced to use it, unless you specify a lower profile.

And same thing for the level. But I understand that I did a mistake.

It will if you do not override it with --ref. That's probably what r0lZ is seeing.
Exactly. Thanks for clarifying what I mean. It is sometimes difficult to explain or understand something when English is not your native language.

But it doesn't enforce vbv so you really can end up breaking the limits. Most people don't notice because they encode with low average bitrates seldomly leading to problematic spikes and because devices don't straight out reject such streams - unlike with streams using too big DPB or too high level flag.

Using --level without --ref is handy because you don't have to calculate yourself, x264 does all the work.
Nicely expressed. Thanks.

VideoFanatic

11th October 2014, 17:22

Sometimes I need to set a Profile such as in MeGUI where's there's no Auto option for AVC Profile. You have to choose one. It lets you set auto for the level but not the profile.

LoRd_MuldeR

11th October 2014, 17:47

OK, I understand now that the preset enforces some specifications. Until now, I thought that its only purpose was to allow more or less time to the encoder. Of course, with less time, the encoder cannot use the most sophisticated compression algorithms, and use the simplest ones, corresponding for example to baseline for ultrafast.

Profiles and Levels say absolutely nothing about how much time the encoder uses for something. They also say absolutely nothing about which "algorithms" the encoder uses (or not uses). The standard/specification doesn't care about the internals of the encoder at all! What the standard does say is: If a certain H.264 decoder supports Profile X, then we can rely on the fact that this decoder supports all H.264 features that are allowed in Profile X, but we can not rely on anything beyond Profile X. Consequently, if a H.264 encoder wants create a H.264 stream that is supposed to work with a decoder that is capable of the Profile X, the the encoder must restrict itself to using only features that are allowed in Profile X.

In other words: If we want to create a H.264 stream that works with all "High" Profile decoders out there, then we must configure our encoder to use only "High" Profile features. If we want to create a H.264 stream that works with all "Main" Profile decoders out there, then we must configure our encoder to use only "Main" Profile features. And if we want to create a H.264 stream that works with all "Baseline" Profile decoders out there, then we must configure our encoder to use only "Baseline" Profile features. Note that the "Main" Profile features are a subset of the "High" Profile features. And that the "Baseline" Profile features are a subset of the "Main" Profile features - at least as far as the feature supported by x264 are concerned.

Once again I suggest to look at this table:
http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Profiles

detmek

11th October 2014, 18:49

No, that's wrong. If your stream would normally come out as Level 5.1, but you enforce Level 4.1 by adding "--level 4.1" (while keeping all other settings the same), you stream's Level requirements do not change at all. Only difference is that your stream will now incorrectly be tagged as Level 4.1, while it actually doesn't comply to that Level - it still requires Level 5.1.

Using the "--level" switch does not reduce the number of reference frames. Instead, x264 will print out a warning, telling you that the selected number of reference frames violates the selected Level:
> x264_8bit_x64.exe --crf 18.0 --preset slower --ref 16

ffms [info]: 1920x1080p 0:1 @ 24000/1001 fps (vfr)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64 SlowShuffle
x264 [info]: profile High, level 5.1> x264_8bit_x64.exe --crf 18.0 --preset slower --ref 16 --level 4.2

ffms [info]: 1920x1080p 0:1 @ 24000/1001 fps (vfr)
x264 [warning]: DPB size (16 frames, 130560 mbs) > level limit (4 frames, 34816 mbs)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64 SlowShuffle
x264 [info]: profile High, level 4.2

If you want fewer reference frames, simply use the "--ref" switch! And, if this actually reduced your stream's Level requirements, x264 will set the new (lower) Level automatically!

You are enforcing --ref 16. Try without that.

Edit:

Sneaker_ger already mentioned that, didn't see. Sorry.

r0lZ

12th October 2014, 00:31

Profiles and Levels say absolutely nothing about how much time the encoder uses for something.
I know that. I was referring to PRESETS. Presets are named ultrafast, faster, slower, and so on. They have obviously something to do with the encoding speed. It's probably because they are very badly named that I have made the confusion at the origin of this discussion. Given their name, it seems that they control ONLY the encoding speed. In fact, they control the profile and level, and therefore, indirectly, the speed of the encoding. When you understand that, everything becomes clear. IMO, the next time someone asks you that kind of question, you should begin by explaining that.

The rest of your post above is well known and doesn't need to be repeated. You just use "feature" when I use "algorithm". The real difficulty is to understand HOW to configure the encoder to obtain a specific profile and level. As I understand now, it's normally NOT via the profile and level options, but mainly via the preset option, at least when the encoder is x264. It's not really intuitive.

In the wiki article, they explain the profile and level concepts, but there is nothing about the presets. It's normal, since the article is about h264 and not about the syntax of x264, but that doesn't help me. I have perfectly understood the profile and level concepts, and I know now that the preset is something made to simplify the selection of the profile and level. I want to understand now how the presets determine specific profiles and levels (if the input file supports them). Currently, I am forced to use a preset without knowing what will be the profile and level used by the encoder, and therefore I can't be sure that my TV will support it. It's why I have usually specified also the profile and the level. Now, if I don't specify them any more, what profile I must use remains a mystery. I will certainly find the solution on the internet, since now I know what I must search. Thanks for having clarified that point.

Groucho2004

12th October 2014, 00:45

In the wiki article, they explain the profile and level concepts, but there is nothing about the presets.
If you use "--fullhelp" on the x264 commandline (and ideally redirect it to a file), you'll see the various parameters for each preset.

r0lZ

12th October 2014, 08:43

OK, thanks!

LoRd_MuldeR

12th October 2014, 12:45

In the wiki article, they explain the profile and level concepts, but there is nothing about the presets.

Profiles and Levels are defined by the H.264 specification. At the same time, there are no such things as "Presets" in the H.264 standard. So, of course, you won't find it in the Wiki article about H.264/AVC.

Presets are simply a convenience function in the x264 encoder software. There is no direct relation between Presets and Profiles/Levels. Though, the Preset may influence, indirectly, at which Profile/Level your stream comes out.

You need to carefully distinguish between properties of the resulting H.264 bitstream, such as Profiles and Levels, and options of a specific H.264 encoder software, such as x264's Preset system.

No, that's wrong. If your stream would normally come out as Level 5.1, but you enforce Level 4.1 by adding "--level 4.1" (while keeping all other settings the same), you stream's Level requirements do not change at all. Only difference is that your stream will now incorrectly be tagged as Level 4.1, while it actually doesn't comply to that Level - it still requires Level 5.1.

Using the "--level" switch does not reduce the number of reference frames. Instead, x264 will print out a warning, telling you that the selected number of reference frames violates the selected Level:
> x264_8bit_x64.exe --crf 18.0 --preset slower --ref 16

ffms [info]: 1920x1080p 0:1 @ 24000/1001 fps (vfr)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64 SlowShuffle
x264 [info]: profile High, level 5.1> x264_8bit_x64.exe --crf 18.0 --preset slower --ref 16 --level 4.2

ffms [info]: 1920x1080p 0:1 @ 24000/1001 fps (vfr)
x264 [warning]: DPB size (16 frames, 130560 mbs) > level limit (4 frames, 34816 mbs)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64 SlowShuffle
x264 [info]: profile High, level 4.2

If you want fewer reference frames, simply use the "--ref" switch! And, if this actually reduced your stream's Level requirements, x264 will set the new (lower) Level automatically!
You are enforcing --ref 16. Try without that.

That was only one example to demonstrate that "--level X" does not actually/reliably make your stream conform to Level X. There may be some specific cases where it works as you expect, but in general it does not! All that "--level" really does, in general, is tagging your stream with Level X - regardless of whether it conforms to that Level or not. This is in contrast to "--profile X", which enforces the restrictions of Profile X and thus guarantees your stream will conform to Profile X.

Keep in mind that the required Level also depends on various properties of your input file, such as the resolution and the frame-rate! Those are things which x264 has absolutely no control over, because they are properties of the source video that x264 cannot influence! This means that each source video unavoidably has a certain minimum Level that it conforms to and it will never be able to conform to an even lower Level - regardless of what encoder settings you are going to use. Consequently, it can easily happen that your are setting "--level X", but form the properties your individual source file (resolution, frame rate, etc) alone it is already clear that this video will never be able to conform to Level X. Bummer!

To make a long story short: Using "--level X" doesn't guarantee that you are going to get a stream that actually conforms to Level X - which means you should be very careful when using that option. I still recommend to not use "--level", but instead do whatever is necessary to make your stream actually conform to Level X. As soon as you have done that, x264 will tag your stream as Level X automatically ;)

r0lZ

12th October 2014, 19:45

Profiles and Levels are defined by the H.264 specification. At the same time, there are no such things as "Presets" in the H.264 standard. So, of course, you won't find it in the Wiki article about H.264/AVC.
Again, you explain something that I know perfectly. It's why I have written " As I understand now, it's normally NOT via the profile and level options, but mainly via the preset option, at least when the encoder is x264." I apologize if I'm not always clear in my requests. But you should understand that a user of Simple x254 Launcher needs the information about how to configure x264, and not any h264 encoder. You gave me information about the profile and levels, but I don't needs that. I just want to know how to configure the x264 encoder to be sure that it will never use a level greater than 4.2, and in the same time, have the best possible compression. Since the preset is the first thing to select (including in your GUI), I suppose that it must be selected carefully. Apparently, it is impossible to obtain that information.
That was only one example to demonstrate that "--level X" does not actually/reliably make your stream conform to Level X. There may be some specific cases where it works as you expect, but in general it does not! All that "--level" really does, in general, is tagging your stream with Level X - regardless of whether it conforms to that Level or not. This is in contrast to "--profile X", which enforces the restrictions of Profile X and thus guarantees your stream will conform to Profile X.I have examined your example and already agreed that if you use incompatible options, of course using --level is absurd. But I did a new test with a short clip (from a BD 1080p at 23.976 fps) with preset veryslow. Without specifying the level, it produces a file at level 5.1 and its final size is 106,963KB. The same file encoded with the same preset and --level 4.2 produces a file tagged as 4.2 (erroneously or not) but this time, its size is 107,207 KB. A very important difference for just a different string (or flag, or whatever) in the header. That's an evidence that specifying --level (I repeat: without other conflicting options), has an influence on the compression, and is not only a trick to patch the header with an intentionally wrong information. So, the --level DOES something, and may be useful. But I agree that I'm not sure that the final file is really compliant to level 4.2. There was no warning in the x264 log, so I assume that it has really created a 4.2 level. Anyway, my TV accepts it, and it's what I want.
To make a long story short: Using "--level X" doesn't guarantee that you are going to get a stream that actually conforms to Level X - which means you should be very careful when using that option. I still recommend to not use "--level", but instead do whatever is necessary to make your stream actually conform to Level X. As soon as you have done that, x264 will tag your stream as Level X automatically ;)
I have tested that too with the same test clip. With any fast preset (from ultrafast to medium), the level automatically used by x264 is 4.0. With slow or slower, it's 5.0, and with veryslow or placebo, it's 5.1. Obviously, the level depends not only of the characteristics of the input file, but also of the preset used to encode it.

My question is essential if you have an hardware player that doesn't support level 5.0 or greater. How can I use one of the slow presets (with that file, or any other), to obtain level 4.1 (or 4.2, or 4.0 - the exact level doesn't matter much, as long as it's less than 5.0). Currently, I need to assume that specifying --level 4.1 is the way to do it, in absence of a better method. It the --level option is really unusable, there must be another way (without specifying tons of complex options) to force the encoder to stay at level 4.0 (since that level is obviously compatible with my input file). I would like to know just that.

sneaker_ger

12th October 2014, 20:48

For high profile with the slow presets just use --level (but never --ref) and set --vbv-maxrate and --vbv-bufsize in accordance with the following table:
https://forum.handbrake.fr/download/file.php?id=50&mode=view

This should suffice 99% of the time. If you make a mistake (for example too high resolution/fps for given level) x264 will print a warning in addition to the profile it uses. But beware, some decoders have additional limitations so you can never be sure it will actually play back perfectly. For example some players require --slices 4 beyond level 4.0 or they further restrict --level 4.1 to --vbv-maxrate 40000 and --vbv-bufsize 30000. Try to think before encoding what level you really need, you can often get away with --level 4.0 or lower without sacrificing quality.

LoRd_MuldeR

12th October 2014, 20:49

I have examined your example and already agreed that if you use incompatible options, of course using --level is absurd. But I did a new test with a short clip (from a BD 1080p at 23.976 fps) with preset veryslow. Without specifying the level, it produces a file at level 5.1 and its final size is 106,963KB. The same file encoded with the same preset and --level 4.2 produces a file tagged as 4.2 (erroneously or not) but this time, its size is 107,207 KB. A very important difference for just a different string (or flag, or whatever) in the header. That's an evidence that specifying --level (I repeat: without other conflicting options), has an influence on the compression, and is not only a trick to patch the header with an intentionally wrong information. So, the --level DOES something, and may be useful. But I agree that I'm not sure that the final file is really compliant to level 4.2. There was no warning in the x264 log, so I assume that it has really created a 4.2 level. Anyway, my TV accepts it, and it's what I want.

This has less to do with "incompatible options". It has more to do with the fact the "--level" does not apply the restrictions of the specified Level, but instead only controls what Level your stream will be tagged with. The exactly same number of reference frames may be compliant with, e.g., Level 4.2 with one source video, but may violate Level 4.2 with some other source clip! That's because the Level does not limit the number of reference frames directly, but the size of the DPB (Decoded Picture Buffer) - which means the max. number of reference frames you can use in a specific Level depends on the resolution source clip. Thus you cannot say "--ref 16" and "--level 4.2" are "incompatible options". They may be perfectly valid!

I have tested that too with the same test clip. With any fast preset (from ultrafast to medium), the level automatically used by x264 is 4.0. With slow or slower, it's 5.0, and with veryslow or placebo, it's 5.1. Obviously, the level depends not only of the characteristics of the input file, but also of the preset used to encode it.

I never said that the required Level depends only on the characteristics of the input file. I said that the characteristics of the input file alone already define a certain minimum Level that your clip can comply to. Your stream will never be able to comply to an even lower Level. Conversely, you can easily make your stream require an even higher Level, e.g. by cranking up the bitrate or by using more reference frames. And it's the latter what happens when you use "Placebo" Preset ;)

detmek

13th October 2014, 06:54

LM, I agree with you that --level does not apply all restrictions for chosen level but it does limit number of of reference frames if those are not specified with --ref x. And level does depend on resolution and number of reference frames according to formula for DPB.http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Decoded_picture_buffering
So, when you encode something with x264 and specify preset and level encoder has all information to automatically set correct number of reference frames or chosen level.

Now, there is a possibility that someone tries to specify level too low for final resolution, lets say level 3.1 for 1080p. And you are right when you say that stream will be flagged as level 3.1 but it won't comply with level 3.1. That is all clear but that does not answers r0lZ's question.

The point is that in his case --level 4.2 does the job as resolution is in the selected level and he only needs to make sure not to exceed maximum number of reference frames with the preset he uses.

r0lZ

13th October 2014, 08:18

Thanks sneaker_ger for the solution! Thanks too to detmek for the clarification, and to LM for having tried to help.

I think I have now exactly what I need. However, I still have a question about the --level argument. I have understood (clearly!) that it flags the stream with a possibly wrong level. It's obviously the case, as detmek wrote, when "someone tries to specify level too low for final resolution, lets say level 3.1 for 1080p". My question is: will it be tagged with the level in the specified --level argument is too high? For example, if I encode my 1080p clip with a fast preset and I specify --level 4.1, the level really used is level 4.0, but the file will be tagged as 4.1. Is it correct? If it's the case, it's a pity.

I need to know that, because ideally I would have preferred a way to specify the maximum level that the encoder should never exceed, regardless of the characteristics of the input file. (And if the input file is not compatible with the specified level OR with a lower level, then in that case, I would like to see a clear warning.) It would be so simple to have an option somewhere to tell something like "My player doesn't support level 5.0 or higher. Do whatever is necessary to create a stream that is compatible with it, but flag it with the level really used to encode the file." In other words, I would like an option to specify the maximum level that is allowed, without having to force a specific level.

OK, it's possible using the table given by sneaker_ger, but it's somewhat difficult to explain to inexperienced users. I would like to implement that option in BD3D2MK3D, and I suppose that I'll have to do it myself, as it seems that that option is missing in x264 itself. Is it correct?

detmek

13th October 2014, 09:41

Stream is always flagged with level specified with --level parameter. In your 1080p --level 4.1 --preset fast example stream will be flagged as 4.1 even though it is level 4.0.

A few years ago there was a discussion about what --level shoud do. The result was that --level flags stream with specified level and limits maximum number of reference frames of selected level is those are not specified with --ref parameter.

LM was right when he said that you should not rely on --level command to do the job. In your case it works because you exceed level 4.2 due to --preset veryslow and --preset placebo use too much of reference frames for selected level and resolution and --level brings those down.

fvisagie

13th October 2014, 11:42

ideally I would have preferred a way to specify the maximum level that the encoder should never exceed, regardless of the characteristics of the input file. (And if the input file is not compatible with the specified level OR with a lower level, then in that case, I would like to see a clear warning.)

Level is not only influenced by input video parameters, but also by the encoding bitrate one chooses. Perhaps the attached shell script can be of some help in your case. This is how I call it currently (in my case I don't need to check for exceeding specified level, but that is easy to add):

rem ...
rem Estimate likely profile to be used with typical encodes
rem -------------------------------------------------------
if /i [%PRESET%] == [ultrafast] (
set PROFILE=ConsBase
) else (
set PROFILE=High
)
if /i [%FFMPDBUG%] == [echo] echo PROFILE=%PROFILE% >&2

rem Calculate H.264 level and maxref
rem --------------------------------
for /f "usebackq tokens=1* delims==" %%i in (`"call h264levl %WIDTH% %HEIGHT% %FPS% %BITRATE% %PROFILE% 1 1 2>NUL"`) do (
if /i [%%i] == [level] (
set LEVEL=%%j
) else if /i [%%i] == [maxref] (
set MAXREF=%%j
) else if /i [%%i] == [x264maxref] (
set XMAXREF=%%j
)
)
if /i [%FFMPDBUG%] == [echo] echo LEVEL=%LEVEL%, MAXREF=%MAXREF%, XMAXREF=%XMAXREF% >&2

rem Check successful parsing
set PARLIST=
for %%i in (LEVEL MAXREF XMAXREF) do (
if [!%%i!] == [] (
if not [!PARLIST!] == [] set PARLIST=!PARLIST!,
set PARLIST=!PARLIST!'%%i'
)
)
if not "%PARLIST%" == "" (
echo Unable to determine video parameter/s %PARLIST%! >&2
goto ERRPAUSE
)
rem ...

r0lZ

13th October 2014, 11:58

I understand. Thanks.

I still need some precisions.

I want to implement a user friendly option to limit the level really used by the encoder, something like a pseudo --max-level option, in BD3D2MK3D. Since BD3D2MK3D is usually used to convert 3D BDs to half-side-by-side or half-top&bottom MKV (therefore in the same resolution than the source video), and that, as far as I know, a 3D BD must be 1920x1080 at 23.976 fps, I suppose that without the --level parameter, x264 produces ALWAYS the same levels as in my tests (4.0, 5.0 or 5.1 for the different presets). So, assuming that the user doesn't force another profile or use fancy options, if he wants, say, limit to max level 4.1 and it selects the veryslow preset, I should use --level 4.1 --vbv-maxrate 62500 --vbv-bufsize 78125. If he selects profile medium (or less), I have nothing to do, since the level is <= 4.1 anyway. Right?

Or is it possible that the default level for 1080p @ 23.976fps is different than in my tests in some cases? If yes, what are the other characteristics of the input file that I should take into account? I have seen that the frame rate matters too, but in 3D BDs it is always 23.976 (sometimes muxed by the user at 25 fps if he uses an audio stream taken from a PAL version of the movie). Anyway, that frame rates are largely below the limits, so I guess they are OK anyway. But are there other characteristics to take into account?

In BD3D2MK3D, there is an option to produce the output video in Full-SBS (3840x1080) or Full-T&B (1920x2160). (I don't like that non-standard 3D modes, but some users have insisted, and I have finally implemented the "Full" option.) According to the tables about the levels, resolutions and frame rates of h264, the minimum level for that width or height is 5.1. But when I encode a full-SBS clip with x264, it uses level 5.0. I suppose therefore that it's not the width and/or height that matters, but the total number of pixels. That seems logical. Is it correct?

Last question. Is the --tune parameter also related directly or indirectly to the level, or can it be freely modified without impact on the level?

Thanks again for your patience.

[EDIT]
Thanks fvisagie for your script. It is always useful to see how peoples do things that you need to understand.

sneaker_ger

13th October 2014, 14:18

So, assuming that the user doesn't force another profile or use fancy options, if he wants, say, limit to max level 4.1 and it selects the veryslow preset, I should use --level 4.1 --vbv-maxrate 62500 --vbv-bufsize 78125. If he selects profile medium (or less), I have nothing to do, since the level is <= 4.1 anyway. Right?
Unless you use the vbv parameters you cannot know the level before you start encoding because you don't know the resulting bitrates. Even 1080p preset medium can exceed level 4.0 through bitrate peaks.

--tune animation can influence the level because it changes the number of reference frames but since --level automatically limits that you don't have to worry about it.

r0lZ

13th October 2014, 15:20

OK, thanks. But x264 prints the level (and profile) that it will use immediately when it starts the encoding, even without vbv parameters. It seems to know it for sure, even before having begun to encode the file (and therefore is can compute it only with basic information such as the width, height and framerate). I wonder how it's possible. Or does it print the maximum level that it could use, but it may use a lower level if it never needs a high bitrate?

At least, the --tune option is easy to use. Good news! :-)

fvisagie

13th October 2014, 16:33

I want to implement a user friendly option to limit the level really used by the encoder

That's not really possible for well-behaved (predictable) software to do. For a given profile, the level depends on the frame size, framerate, bitrate and whether B-frames and B-pyramids are used in encoding (the latter two influence the number of reference frames used by x264, which in turn influences the level). Then, keeping all settings the same but changing the profile may change the level (see the effect on levels of using different profiles at http://en.wikipedia.org/wiki/H264#Levels).

There are too many explicit or implicit user settings that culminate in a given level to expect software automatically to adjust and/or compensate them all in a manner that does not lead to unexpected results or quality degradation.

A more workable approach may be to anticipate the bitrate for a given encode, then from the bitrate (and other settings) to anticipate the resultant level, and to raise an error if that exceeds the specified level. One way to implement that would be:

Encode in way that makes bitrate predictable (either do two-pass encoding or use the vbv parameters as suggested)
Use the script posted above to calculate the level anticipated
Compare that to the specified level and either allow encoding to proceed or halt with an error

This could be done by hand; if you do many encodes it could quite easily be automated in a shell script or program language.

r0lZ

13th October 2014, 16:50

It's for my program, BD3D2MK3D, that I want to do it. It has to be automatic. I will use the vbv parameters, since that seems mandatory.

But still, I don't understand how it is possible that x264 prints the level it will use for a given input file when only the preset has been selected (and NO vbv parameters). It has to evaluate the bitrate given the picture size and the frame rate, and, of course, it knows already the other parameters it will use, since they are determined by the preset. So, if it uses, say, level 4.0 for preset medium and 1080p at 23.976 fps, I don't understand why it should use another level for the same preset and another input file but with the same resolution and frame rate (and, of course, if no other arguments have been provided). Since it cannot analyse the file during a first pass, and it prints the level info BEFORE it begins to encode the file, it must evaluate it with only those parameters: the preset, and the width, height and frame rate of the input file.

If it cannot do that, then the info it prints at the beginning of the encoding may be wrong. As far as I know, it's not the case. I'm puzzled!

detmek

13th October 2014, 17:24

When x264 prints level it does not take bitrate and vbv into calcultaion. It uses resolution, framerate and reference frames. So, encoder might print wrong level if real bitrate is very high and exceeds VBV.

The reason we had no problems so far is because most hardware players have very large bufer and even higher read speed.

sneaker_ger

13th October 2014, 17:35

When x264 prints level it does not take bitrate and vbv into calcultaion.
It does take --bitrate and vbv into consideration AFAIK but of course if you do not specify vbv it can only guess and - like you said - the guess can turn out to be wrong.

r0lZ

13th October 2014, 17:38

Hum, so the info it prints at the beginning may be wrong! Bizarre, but OK, I understand that it cannot be trusted.

Additional question: if, during the encoding, the real bitrate exceeds VBV (and assuming that --level has not been specified), does it tag the file with the wrong level printed before the encoding, or does it tags it with the real level based on the real bitrate? In the first case, that means that usually, the level in the header of the file cannot be trusted by the player. That's incredible IMO.

sneaker_ger

13th October 2014, 17:46

does it tag the file with the wrong level printed before the encoding
Yes, the level is written at the start and will not be corrected if it turns out wrong.

foxyshadis

14th October 2014, 00:04

And you can test this easily: Set a large number of reference frames via "--ref" switch and force a Level that doesn't actually allow that many reference frames (due to DPB size limit) via "--level" switch. Will x264 lower the number of reference frames for you, to make your stream actually comply to the enforced Level? No it doesn't! It only prints a warning message that your stream violates the Level! Does it still still set the enforced Level, despite your stream is violating the Level? Yes.

If you only use the preset option and not --ref directly, that doesn't apply, but only in the command-line:
/* Automatically reduce reference frame count to match the user's target level
* if the user didn't explicitly set a reference frame count. */
If you use libx264 directly, you're trusted to know what you're doing and not set it incorrectly unless you want to.

LoRd_MuldeR

14th October 2014, 01:32

If you only use the preset option and not --ref directly, that doesn't apply, but only in the command-line:
/* Automatically reduce reference frame count to match the user's target level
* if the user didn't explicitly set a reference frame count. */
If you use libx264 directly, you're trusted to know what you're doing and not set it incorrectly unless you want to.

Yes, if you don't use "--ref" at all, then x264 will automatically set the number of reference frames according to Level specified via "--level".

But the point is that x264 does not enforce the restrictions of the Level set via "--level" switch, which is in contrast to "--profile" switch. That's because there are other things, such as the video's resolution and frame-rate, that affect Level compliance. Thus, setting "--level X" may produce a stream that complies with Level X, but it may just as well produce as stream that does not comply with Level X (but would still be tagged, wrongly, as being compliant with Level X).

So again: Why would you explicitly add the "--level" switch? If, by default (that is: without "--level"), your stream came out at a Level this is above the Level that you had intended, then you can not reliably make your stream conform to the intended Level just by adding the "--level" switch! Instead, you should probably figure out what exactly is currently preventing your stream from conforming to the intended Level and fix that. And if, by default (that is: without "--level"), your stream came out at a Level this is even below the Level that you had intended, you can just be happy and don't have to change anything. The only reason to set "--level" explicitly that comes to my mind is when x264 actually got the Level wrong.

fvisagie

14th October 2014, 09:16

Yes, if you don't use "--ref" at all, then x264 will automatically set the number of reference frames according to Level specified via "--level".

The only reason to set "--level" explicitly that comes to my mind is when x264 actually got the Level wrong.

There is another, very important, reason. Due to the subtlety you allude to in the first quote, --level is needed to ensure --ref is set correctly. If the command line omits --level, --ref defaults to 16. I just retested that to confirm.

That means all encodes that need --ref <> 16, and for which the appropriate --level is not specified, will be non-level-conformant.

Therefore, for an encode to conform to whatever level its parameters and bitrate translate to, one needs to:

either specify the corresponding --level (for which one obviously needs to correctly anticipate it)
or to let x264 default the level and to manually specify --ref (for which one needs to correctly anticipate the level in any case)

In other words, to ensure an encode is level-compliant, it seems there is no easier way than having to anticipate the level, and to specify it.

@r0lZ,
In addition, where an encode needs to be constrained to a maximum level as is the case with you, you'd also need to compare the specified maximum level to the anticipated encoding level as suggested in post #34 (http://forum.doom9.org/showthread.php?p=1696780#post1696780).

For what it's worth, attached is an example encoding batch file that uses h264levl.bat posted above to anticipate the encoding level. Note that this batch file does not currently constrain the encoding level as you want to, although that would be trivial to add as mentioned.

r0lZ

14th October 2014, 09:33

Yes, the level is written at the start and will not be corrected if it turns out wrong.
Damn! The management of the level in x264 is rather complex! Not only it is impossible to limit the encoder to a maximum level (without forcing a specific level), but even when you let x264 decide what level to use, it can tag the file with a wrong level! A very big limitation, plus a big bug. It's a pain for the users.

So, I have no other choice, for BD3D2MK3D, than to add 2 options:

1. Leave x264 decide what level to use, and it may produce streams incompatible with your hardware if you use a slow preset. In addition, it may even tag the stream erroneously. IMO, that's the solution to use when the user encodes with any fast or medium preset, because the resulting level is 4.0 (or perhaps 4.1 or even 4.2) for 1080p, and that levels are compatible with most players. That can be easily explained in a help dialog.

2. Or force a specific level, such as 4.2, with the --level and --vbv-* arguments, and in that case, it is possible that the specified level is actually too high for the selected preset and the stream may have been encoded with a lower level, but will be tagged erroneously with the specified level. Therefore its compatibility with the players will be less good without any good reason. But at least, it is possible to encode with any slow preset without breaking the compatibility with many hardware players.

There is obviously no good solution, but I will have to implement the two options above, because IMO the worst solution is to encode blindly at any level without the possibility for the user to be sure that its file will be compatible with his player.

Thanks to everybody. I know now the limitations and problems of x264, and how to offer a solution to the casual user of BD3D2MK3D, even if the solution is far to the perfection.

Of course, if there is still a problem in the methods explained above, please let me know.

[EDIT] Oh, yes, there is also the possibility to anticipate the level, as explained by fvisagie above. But since the bitrate depends mainly of the "complexity" of the scenes in the input file, and that it is impossible to analyse that with a batch file, the result of any anticipation based on the analysis of the input file and parameters may not be totally accurate. Given the fact that all files encoded by BD3D2MK3D are 1080p @ 23.976 fps, I suppose that I can simply assume that the levels are 4.0 (or a bit above in case of bitrate peaks) for any preset <= medium, and 5.0 or 5.1 (or a bit more) for the slow presets. There is normally no need to constraint the level when the user encodes with a fast preset, so finally, in the case of BD3D2MK3D, it is possible to offer a relatively simple solution.

detmek

14th October 2014, 09:35

My encoding experience is that I use 4 reference frames for every encode, no matter what preset I use. 4 reference frames are maximum for level 4.1 and 1080p30. That way I don't have to worry that strem will exceed level 4.1 but it will flagged with lower level if resolution is lower.

Kurtnoise

14th October 2014, 09:42

fvisagie

14th October 2014, 09:49

So, I have no other choice, for BD3D2MK3D, than to add 2 options:

1. Leave x264 decide what level to use, and it may produce streams incompatible with your hardware if you use a slow preset. In addition, it may even tag the stream erroneously. IMO, that's the solution to use when the user encodes with any fast or medium preset, because the resulting level is 4.0 (or perhaps 4.1 or even 4.2) for 1080p, and that levels are compatible with most players. That can be easily explained in a help dialog.

2. Or force a specific level, such as 4.2, with the --level and --vbv-* arguments, and in that case, it is possible that the specified level is actually too high for the selected preset and the stream may have been encoded with a lower level, but will be tagged erroneously with the specified level. Therefore its compatibility with the players will be less good without any good reason. But at least, it is possible to encode with any slow preset without breaking the compatibility with many hardware players.

There is obviously no good solution, but I will have to implement the two options above, because IMO the worst solution is to encode blindly at any level

The third (and in my view more suitable) option would be for your program to anticipate the resultant encoding level as suggested above, to compare that to the maximum specified, to proceed if within limits or, in case not, to terminate with a level warning (and optionally to suggest trying a lower bitrate, that being directly under user control). In all cases where your encode does proceed, you would need to specify the --level as anticipated to ensure --ref is set to the corresponding level limit.

fvisagie

14th October 2014, 09:54

But since the bitrate depends mainly of the "complexity" of the scenes in the input file, and that it is impossible to analyse

That is not the case for constant bitrate encoding (i.e. either two-pass encoding or using the vbv parameters). If your program generates constant bitrate encodes, you can safely use the specified bitrate to anticipate and/or limit the resulting encoding level.

r0lZ

14th October 2014, 10:12

Level is not a tag in the bitstream...Anyway, there are some tools to override this.

What do you mean by compatibility with many hardware players ? Why not put these switches --bluray-compat --vbv-maxrate 40000 --vbv-bufsize 30000 --level 4.1 in your command line then ? That should be fine for all players.
Can you give a link to one of these tools?

I don't want compatibility with bluray players, but with any player (hardware or software) that any user may have. As explained in a post earlier in this thread, a good example of this is my TV. It supports level 4.2 or below, but not 5.0. But I want to be able to use the slow presets, because they are supposed to compress better, but limit them to a level that is compatible with the target player of the user. (I can't limit to 4.1 anyway, because many users of my program want 5.0 or more.)

r0lZ

14th October 2014, 10:19

The third (and in my view more suitable) option would be for your program to anticipate the resultant encoding level as suggested above, to compare that to the maximum specified, to proceed if within limits or, in case not, to terminate with a level warning (and optionally to suggest trying a lower bitrate, that being directly under user control). In all cases where your encode does proceed, you would need to specify the --level as anticipated to ensure --ref is set to the corresponding level limit.
Yes, I know that it's a good solution, but BD3D2MK3D has several limitations. For example, I can't launch an encode just to verify that the level doesn't exceed the limit, because the program extracts the AVC, MVC, audio and subtitle streams to a "project" directory, and builds a batch file to encode. When the batch file is launched by the user, there is no way to interrupt it, and if finally, the level really used exceeds what the user wants, he will have to redo everything from the beginning, including the long demux process. Well, he can also edit the batch file and some other files, but I want a solution for inexperienced users, and editing a batch file is something really too hard for many peoples! ;-)