x264: suggested number of ref frames?

[TECK]

I read quite a bit on the x264 settings to encode a movie.
Here it is an an example how I lower the bitrate of a file encoded initially with --crf 16:
program --level 4.1 --pass 2 --bitrate 5195 --stats ".stats" --output "output" "input"

I was wondering if anyone could tell me if I should use --ref 5, instead of default value and what are the benefits. Example of encoded file, with the above settings:
Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.1
Format settings, CABAC : Yes
Format settings, ReFrames : 4 frames
Muxing mode : Header stripping
Codec ID : V_MPEG4/ISO/AVC
Duration : 2h 23mn
Bit rate : 5 195 Kbps
Width : 1 280 pixels
Height : 696 pixels
Display aspect ratio : 16:9
Frame rate : 23.976 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.243
Stream size : 5.10 GiB (87%)
Writing library : x264 core 120 r2164 da19765
Encoding settings : cabac=1 / ref=3 / ... / aq=1:1.00
Language : English
Default : Yes
Forced : No

There is another thing that is confusing, Why the ReFrames show as 4, when the --ref is clearly marked as 3 into encoding settings?

Thank you.

[nurbs]

Depends. You are probably off best if you just let x264 handle the number of reference frames using the --preset and --tune commands. In your example if you use 5 refs and your content is 1080p you break the restrictions of level 4.1 which can't happen if you stick to presets and tunings, so you save yourself trouble.
More reference frames give you better compression, how much depends on the content. Cartoons usually benefit more than real world footage and run into diminishing returns later. The presets are chosen to give a good ratio of compression gain to speed loss, so even if you don't want to use them and instead change specific settings manually I'd recommend you still use them as guidelines.

IIRC using b-pyramid requires the same DPB as using 4 reference frames, so if you haven't turned it of that's the reason.

[TECK]

Thanks for the good info. So I should use a preset and tune on a regular basis? I usually have the --preset set to medium. I'm going to try encoding with --tune film and see what is different in settings.

[LoRd_MuldeR]

Basically the Presets control "Speed -vs-quality", while the Tunings tweak x264 for a specific kind of source. You should use both of them.

Use the slowest possible Preset you can afford. And use the Tuning that matches your individual source.

If you need to encode for a specific H.264 Profile and/or H.264 Level, you can use "--profile" and/or "--level" to enforce the required restrictions.

Last but not least, using "--preset medium" does nothing, as that is just the default ;)

[hello_hello]

TECK, do you use MeGUI?

If you want to see what changes when using different speed presets etc, open MeGUI's x264 encoder configuration, load the defaults, then select "advanced options". The encoder settings MeGUI displays under the various tabs change according to the preset.

Tune film will get you "deblock=1:-1:-1"

[TECK]

Use the slowest possible Preset you can afford. And use the Tuning that matches your individual source.
Based on my little experience, I noticed much better results if I first encode a Bluray with --crf 16 than I re-encode and lower the video bitrate to a specific value in order to keep the file size in line. For example, there are movies that output 5000 as bitrate while others 7000. I like to keep my encoded files around a 5195 bitrate, is the best number for my Sony 55inch TV. So if a Bluray rip has a 7000 bitrate, I re-encode it with default settings and --level 4.1 --pass 2 --bitrate 5195. That combination gives me stunning results and a great file size, with virtually no quality loss from the original rip.

On the other hand, if I can obtain even better results with a tuned setting and preset, I will definitely use it. I will experiment and see how it goes. I guess what I'm trying to determine is if you guys define a specific tune for each movie or just go with the default settings.

Last but not least, using "--preset medium" does nothing, as that is just the default ;)
Ya. :) So far I always used the default, which explains the command line posted into OP:
program --level 4.1 --pass 2 --bitrate 5195 --stats ".stats" --output "output" "input"

[LoRd_MuldeR]

Based on my little experience, I noticed much better results if I first encode a Bluray with --crf 16 than I re-encode and lower the video bitrate to a specific value in order to keep the file size in line.

2-Pass and CRF basically use the same RC algorithm. Consequently the difference between a CRF and a 2-Pass encode of the same file size (i.e. same average bitrate) is negligible.

The difference is that with 2-Pass you can predict the final file size beforehand. With CRF you can't (but you safe one pass).

So if you need to hit a specific file size, simply run a "normal" 2-Pass encode. Otherwise use CRF with the lowest possible CRF value that satisfies your eyes...

[TECK]

I totally agree with you. However, my goal is not to obtain a specific size but rather a specific bitrate that is optimal for my eyes and TV setup. The best way for me to determine the bitrate from a "clean" encode is by using a --crf 16. If the bitrate is lower than 5195, than I leave the movie untouched. If it is higher, I run another 2 pass encode and lower it to 5195. In this way, I get the best of both worlds.

[vdcrim]

If the bitrate is lower than 5195, than I leave the movie untouched. If it is higher, I run another 2 pass encode and lower it to 5195.
It would be faster to run a first pass with bitrate 5195 and check the 'final ratefactor' reported. If it's higher than 16, do a second pass. If it's lower, encode with CRF 16.

[LoRd_MuldeR]

I totally agree with you. However, my goal is not to obtain a specific size but rather a specific bitrate that is optimal for my eyes and TV setup. The best way for me to determine the bitrate from a "clean" encode is by using a --crf 16. If the bitrate is lower than 5195, than I leave the movie untouched. If it is higher, I run another 2 pass encode and lower it to 5195. In this way, I get the best of both worlds.

If you first run a CRF encode with a specific CRF value and then run a 2-Pass encoder with a target bitrate lower than what your CRF encode came out, this means that you are effectively re-doing your CRF encode with a higher CRF value. If that higher CRF value still gives acceptable quality, you could as well have started with the lower CRF value and skip the additional 2-Pass encode.

[nm]

It would be faster to run a first pass with bitrate 5195 and check the 'final ratefactor' reported. If it's higher than 16, do a second pass. If it's lower, encode with CRF 16.

That's a good idea if input is fast, so that the fast first pass is not hindered too much.

If input is slow compared to encoding (because of filtering or when resizing from HD to SD), it might be faster in the long run to encode first with --slow-firstpass --pass 1 --crf 16, and only run a second pass if that encode ends up being too big. That way you get away with only one pass most of the time and occasionally need to run a second one.

Personally I'd just raise CRF to 17 or 18 for all encodes. Noisy sources might need special treatment to keep the bitrate sane, but IMO using too low bitrate is worse than denoising.

[TECK]

It would be faster to run a first pass with bitrate 5195 and check the 'final ratefactor' reported. If it's higher than 16, do a second pass. If it's lower, encode with CRF 16.
That is pretty neat, I will try it next time. The time is not really an issue as I usually encode the movie during the night, with an Intel I7-990X proc.

If you first run a CRF encode with a specific CRF value and then run a 2-Pass encoder with a target bitrate lower than what your CRF encode came out, this means that you are effectively re-doing your CRF encode with a higher CRF value. If that higher CRF value still gives acceptable quality, you could as well have started with the lower CRF value and skip the additional 2-Pass encode.
Ya, I understand. Since I'm not as experienced as you guys, the method I'm using was what I thought would be best to suit my needs. I will try your suggestions on the next video encode. :)

Personally I'd just raise CRF to 17 or 18 for all encodes. Noisy sources might need special treatment to keep the bitrate sane, but IMO using too low bitrate is worse than denoising.
Just curious, what is your average bitrate with CRF 18? For myself, it varies a lot from movie to movie, with CRF 16. Hence the reason why I redo the encoding for certain movies. I had movies that output a 4500 bitrate while others went over 7000. Go figure. :)

[Didée]

I like to keep my encoded files around a 5195 bitrate, is the best number for my Sony 55inch TV.
Basically nonsense. There are films that will turn out embarrasingly bad at that bitrate. And there are other films for which that bitrate is totally overkill.

There is no "this is the good bitrate" because each movie is different. The "good" bitrate is dictated by the content, and is different for each movie.

(On a series I backuped recently, some episodes would turn out at 12000 kbps (!), at CRF 20, and at 720p (!!), when not doing dedicated grain-easening filtering. )

[nm]

Just curious, what is your average bitrate with CRF 18?

I'm using CRF 21 for backing up PAL MPEG-2 DVB captures. Quality of the broadcasts is so bad that using a lower CRF doesn't make any visual difference. My bitrates vary from 500 kbps to 2500 kbps depending on content. For bobbed 50p, the range is 1000 kbps to 3500 kbps, and I tend to get 1:2 or worse compression ratios compared to the original interlaced broadcast, sometimes exceeding the original size. Encoding as interlaced would be a bit more efficient, but I like to use TGMC for deinterlacing (Thanks Didée! :)).

I also denoise most stuff with MDegrain2. It kills some details (rain, other small moving particles), but I don't like to spend disk space for storing noise.

[shae]

For SD resolutions with a higher number of ref frames, is it safe to assume it will be compatible with TV/streamer playback as long as the L4.1 buffer memory constraints are kept?
(I'm not considering increasing the number from implicit default, but decreasing, if needed.)

[stax76]

TECK, do you use MeGUI?

If you want to see what changes when using different speed presets etc, open MeGUI's x264 encoder configuration, load the defaults, then select "advanced options". The encoder settings MeGUI displays under the various tabs change according to the preset.

Tune film will get you "deblock=1:-1:-1"

You can also get it with --help, in staxrip at: Main Menu > Help > Apps > x264

If you want it as browser bookmark the megui wiki or pastebin could be used.

[hello_hello]

For SD resolutions with a higher number of ref frames, is it safe to assume it will be compatible with TV/streamer playback as long as the L4.1 buffer memory constraints are kept?
(I'm not considering increasing the number from implicit default, but decreasing, if needed.)

I think the worst that can happen is you'd lose compatibility with older devices that only support level 3 or 3.1 etc for standard definition, but other than that I can't imagine why it'd matter.
The players in my house all support level 4.1 so I use it for everything, regardless of resolution.

[Motenai Yoda]

I'd like to use the lower level for a resolution/framerate, usually more than 5 reference don't give sensible more compressibility

[shae]

In theory L4.1 compatibility should mean exactly that, but who knows what happens in practice.
Though I suppose modern hardware players are more accommodating than past ones, probably because they use common software.

[hello_hello]

I had assumed Motenai Yoda was referring to the number of reference frames in relation to encoding speed rather than playback problems.

If you specify one of the very slow presets that in turn sets increases the number of reference frames, when also specifying a level the encoder will limit reference frames according to the resolution and what the level permits.

I mostly use the Slow preset myself, or sometimes Slower, and they set reference frames to 5 and 8 respectively, so I don't think about them much.

[benwaggoner]

Basically nonsense. There are films that will turn out embarrasingly bad at that bitrate. And there are other films for which that bitrate is totally overkill.

There is no "this is the good bitrate" because each movie is different. The "good" bitrate is dictated by the content, and is different for each movie.
Put another way, if you want a consistent video quality across a broad range of content types, using CRF instead of bitrate will give you much more consistent video quality. If you've found a certain bitrate works well for you, perhaps you tested on similar kinds of content? A noisy move and clean CGI using CRF 20 will come out with ballpark similar visual quality but very different bitrates.

If you have a max file size you want to hit, you could encode with CRF by default, and then do a 2-pass encode for files that wind up too big. If it was x265, you could do a 1st pass as CRF and then only do a 2nd pass to reduce file size, but I don't believe all the required parameters have been accepted into the mainline x264.

[Motenai Yoda]

I had assumed Motenai Yoda was referring to the number of reference frames in relation to encoding speed rather than playback problems.
Err nope.
I was referring exactly about playback issues, a low budget/power device can play even an h264 stream @ level 5.1 or unrestricted, but fail with a 9+ reference one

[hello_hello]

Err nope.
I was referring exactly about playback issues, a low budget/power device can play even an h264 stream @ level 5.1 or unrestricted, but fail with a 9+ reference one

Isn't that because many players don't check the level, only that a video conforms to what it can decode?

For example, if you encode at 720p (24fps) with the default x264 settings and Level 5.2, chances are a player supporting Level 3.1 will play it as long as the bitrate remains low enough, because other than that, I'm pretty sure it'd comply with Level 3.1.