x264 efficiency & Presets

[stuartmp]

Hi All,

I have had a problem that has been bothering me for awhile now and I was hoping someone with a lot more experience on the topic could help me out.

I want to understand Encoding efficiency vs Filesize

For example

I resonantly encoded a Blu-Ray lossless video stream which was about 25GB using MeGUI and a x264 encoder.

Encoder settings were
- Constant Quality = 20
- Preset = Slow
- Tuning = Film
- AVC Profile = High
- AVC Level = Unrestricted

Output file size was 6,975 Mb (Not to bad)

However I thought if I was prepared to wait a little longer maybe I could get the same quality but at a smaller file size.

So I encoded the original lossless video stream again.

this time I used the settings

- Constant Quality = 20
- Preset = Slower
- Tuning = Film
- AVC Profile = High
- AVC Level = Unrestricted

So I only changed the preset from slow to slower everything else remained the same.

Now the output file size was 7,260 Mb

I don't understand, I would have expected a smaller file.

Can someone please explain to me why the file size increased.

I would have thought that with my Constant Quality set to 20 and only changing the Preset from slow to slower the second output stream would have had the same quality as the first but the file size would be smaller.

Thanks

Stuart

[vivan]

What MeGUI calls "Constant Quality" is not a constant quality. No encoder could actually produce constant quality with different settings. It's actually crf - constant ratefactor, which is something close to quality but only as long as settings are the same. That means that if you don't change settings and only change this value - the resulting size/bitrate/quality will always go up (if you're decreasing it) or down (if you're increasing it). However if you do change settings and keep same crf - the resulting quality will be different.
So, comparing encodes with the same crf and different settings is wrong. If you want to compare different settings you either have to match bitrate (and compare quality) or match quality (not realistically possible since it's subjective, there're objective ways to measure quality but they don't perfecty represent quality from the human point of view) and compare bitrate.

[stuartmp]

Thanks, for that vivan I really appreciate it.

That clear up the understanding of Constant Quality ('crf - constant rate factor') for me, but also raises another two questions.


Q1. If I was to play around with changing the 'crf - constant rate factor' for the encoding of the second video stream so that the final file size was the same and hence the bit rate was basically the same.

For example

- Constant Quality = 21.5
- Preset = Slower
- Tuning = Film
- AVC Profile = High
- AVC Level = Unrestricted

Would the actual video quality of the newly encoded second video stream be better, worst or the same as the video encoded with the first encoding settings?

- Constant Quality = 20
- Preset = Slow
- Tuning = Film
- AVC Profile = High
- AVC Level = Unrestricted

------------------

Q2. How do you compare compression efficiency and video quality between two or more video streams when the bit rate is very close to being the same?

[Sharc]

Well, at same file size I would expect the slower setting to produce slightly better results.
Metric? There are SNR and PSNR metrics, but at the end use your eyes. You may discover differences with slower settings, inspecting frame by frame..... But often it is not easy to tell which one is really (visually) "better".

[jpsdr]

I think, to be more accurate, crf produce a constant quality, but you don't target a specific quality. Crf means that encoder according the complexity of scenes, will increase or decrease bitrate to keep the quality the same on the whole video. As vivian said, if you make 2 encodes of the same file, with same value of crf but different value on other parameters, your both file will have a constant quality, but they won't have the same quality.
So, i repeat, keep in mind, crf produce a constant quality, but you don't target a quality. I've also been through this and in first time made this error before truly understanding. Wiki h264 page is also somehow faulty. What's said about crf parameter is not wrong, but without more details, it can produce false assumption.

[detmek]

No, CRF is not constant quality, it is constant rate factor. If you use same CRF with different settings you will get different quality. Also, Constant Rate Factor produces "constant" visual experience during playback, its not that every frame is of the same quality as next or previous frame. So, CRF is NOT constant quality and should not be called that. That just creates general confusion.

[stuartmp]

Thanks detmek...

Ok I get 'crf' now, can anyone help me with my other 2 questions?

[Warperus]

Q1 Would the actual video quality of the newly encoded second video stream be better, worst or the same as the video encoded with the first encoding settings?
Crf is supposed to be tweaked to provide equal quality. Unfortunately, in reality it may miss both ways - better quality or worse quality. Difference is generally not too big though.

Q2. How do you compare compression efficiency and video quality between two or more video streams when the bit rate is very close to being the same?
One cannot compare compression efficiency between different video streams. Quality metrics tend to compare video quality degradation from source to encoded stream. Numbers then go into table/graph and then comparision speculates about numbers. Different sources are just that, different sources. People believe that similar sources will be processed with similar efficiency...

[stuartmp]

Thanks for that Warperus, but I think I need to clarify my second question.

Q2. How do you compare compression efficiency and video quality between two or more video streams created from the same source, when the bit rate is very close to being the same and all you have changed is a few settings?

For example....

--ref 5 to --ref 8
--subme 8 to --subme 9
--rc-lookahead 50 to --rc-lookahead 60

Keeping everything else the same......

Is it just a personal visual things or is there software or something else out there that can tell you which of the two streams matches the original source the closest.

[vivan]

Q1. If I was to play around with changing the 'crf - constant rate factor' for the encoding of the second video stream so that the final file size was the same and hence the bit rate was basically the same.

...

Would the actual video quality of the newly encoded second video stream be better, worst or the same as the video encoded with the first encoding settings?This is the goal of presets - to give better quality at the same bitrate. I believe that x264 presets are quite good at it.
You can actually use 2-pass bitrate encoding to match the bitrate - crf and 2-pass encoding uses same rate control algorithm, so if you encode using -crf, and then encode using 2-pass bitrate mode to match the first file, their quality will be very similar.

Q2. How do you compare compression efficiency and video quality between two or more video streams when the bit rate is very close to being the same?Visualy, which could be quite hard if quality is very close.

There're objective metrics (SSIM, etc.) that give precision results, however they don't always correlate with what human sees. E.g. they usually favor blurry video with lower amount of errors, while sharper video actually looks better.

However with the same encoder (x264) and same aq and psy settings (IIRC presets don't touch them) using ssim could make some sense.

[Asmodian]

Only because it sometimes helps to have extra answers with different phrasing.

Q1. If I was to play around with changing the 'crf - constant rate factor' for the encoding of the second video stream so that the final file size was the same and hence the bit rate was basically the same.

For example

- Constant Quality = 21.5
- Preset = Slower
- Tuning = Film
- AVC Profile = High
- AVC Level = Unrestricted

Would the actual video quality of the newly encoded second video stream be better, worst or the same as the video encoded with the first encoding settings?

- Constant Quality = 20
- Preset = Slow
- Tuning = Film
- AVC Profile = High
- AVC Level = Unrestricted

------------------


The first one, preset Slower, would be better quality.
(note: slow v.s. very slow will never really be 1.5 crf apart to give the same size)

Q2. How do you compare compression efficiency and video quality between two or more video streams when the bit rate is very close to being the same?

You do not. You have to match the bit rate using 2-pass as vivan stated.

SSIM could work as well but isn't very reliable and personally I wouldn't put much stock in small changes in SSIM.

Don't over-think it and approach the question differently. Don't pick crf and then try to decide if slower presets might be worth it based on size/quality improvements. Too many variables get mixed together and it is hard to determine anything, especially across different sources. Pick the slowest preset you are willing to use first and then tune crf for the quality/size you desire.

[jpsdr]

No, CRF is not constant quality, it is constant rate factor. If you use same CRF with different settings you will get different quality.
Nice, it's exactly what i've said.

Also, Constant Rate Factor produces "constant" visual experience during playback.
That's what you can commonly call a "constant quality".

[detmek]

Right. Sorry about the first sentence. I misread your post. You ment two encodes will have constant quality across video per video but videos will be of different quality if you compare it.

About second part, I didn't express myself correctly. CRF does not produce constant visual quality, it tries to produce it. One source can be encoded with good quality using CRF X and settings Y but those settings may produce lower quality video for some other source. For example, some settings may work for clean source but produce visible artifacts for grainy source, or film vs cartoon source.

In general, you are right and this might be nitpicking.

[Warperus]

is there software ... that can tell youThere is software, but I'm too lazy and greedy to find one suitable.
But if you are talking about x264, you can simply make it compute metrics with its own options:
--psnr Enable PSNR computation
--ssim Enable SSIM computation

[jpsdr]

CRF does not produce constant visual quality, it tries to produce it. One source can be encoded with good quality using CRF X and settings Y but those settings may produce lower quality video for some other source.

You pinpoint an interesting point. The only "constant" thing i've read about crf, is that you can expect similar quality between two different sources only if you encode with the exact same settings. But indeed this rule may be "true" only if the nature of the sources is somehow similar, or not too much far away.
Even with the exact same parameters, you may not finaly have the same quality between a clean absolutely noiseless japanese anime, and a very grainy/noisy very old film live footage...:rolleyes:

The wiki x264 page which explains the settings of x264, and the crf part realy need to be updated and explained more precisely, because actualy it will realy produce confusion, as i've been trapped also at first.

[pcordes]

You pinpoint an interesting point. The only "constant" thing i've read about crf, is that you can expect similar quality between two different sources only if you encode with the exact same settings. But indeed this rule may be "true" only if the nature of the sources is somehow similar, or not too much far away.
Even with the exact same parameters, you may not finaly have the same quality between a clean absolutely noiseless japanese anime, and a very grainy/noisy very old film live footage...:rolleyes:

The wiki x264 page which explains the settings of x264, and the crf part realy need to be updated and explained more precisely, because actualy it will realy produce confusion, as i've been trapped also at first.

You're talking about
https://en.wikibooks.org/wiki/MeGUI/x264_Settings? The whole page needs a rewrite, since it doesn't mention presets at all. But yeah, the crf section could use an update. Go ahead if you have any good ideas for how to write something short but good.

My feeling is that which if any settings you tweak beyond choosing a preset is very specific to your content and use-case. (streaming vs. offline encoding, barely-enough bitrate vs. beyond transparent, etc.)

One thing I've noticed x264 doesn't have a preset for is superfast no-cabac. ultrafast (me=0) is HUGE step down in RD compared to superfast-nocabac (me=1), for something like crf10 scratch files while video editting. (while you try stuff, before going back and redoing the whole chain with no lossy temp files). At very high bitrates, cabac takes a lot of the total CPU time (esp. for decoding, and for video editting that's relevant.)

[stuartmp]

Hi All,

Thanks for all the input into this thread.

The thing that annoys me the most with CRF encoding is.

Depending on the size of the original source file and original bit rate you occasionally end up with an encoding that is not what you expect.

The files size is either much smaller than usual or much larger that expected and who wants to wait another 3-4 hours for another encoding to finish at a new CRF when you still have no idea of the output.

For example
1.
- Source video only file size 25.3 Gb - Bit rate- 25.6 Mbps
- Encoded file size 6.1 Gb -- Bit rate 6 652 Kbps

2.
- Source video only file size 33.1 Gb - Bit rate- 29.7 Mbps
- Encoded file size 2.7 Gb -- Bit rate 2 590 Kbps


both done with the same Encoder settings
--preset slow --tune film --crf 20.3 --ref 6 --subme 9 --lookahead-threads 1 --rc-lookahead 60




I think I may give 2 pass encoding a shot and set my bit rate VBR to 6500 kbps or something.

This way I assume I will get at least a CRF of under 20 on your typical film preset.

Does anyone else have the same dilemma?

[Sharc]

Hi All,

Thanks for all the input into this thread.

The thing that annoys me the most with CRF encoding is.

Depending on the size of the original source file and original bit rate you occasionally end up with an encoding that is not what you expect.

The files size is either much smaller than usual or much larger that expected and who wants to wait another 3-4 hours for another encoding to finish at a new CRF when you still have no idea of the output.

For example
1.
- Source video only file size 25.3 Gb - Bit rate- 25.6 Mbps
- Encoded file size 6.1 Gb -- Bit rate 6 652 Kbps

2.
- Source video only file size 33.1 Gb - Bit rate- 29.7 Mbps
- Encoded file size 2.7 Gb -- Bit rate 2 590 Kbps


both done with the same Encoder settings
--preset slow --tune film --crf 20.3 --ref 6 --subme 9 --lookahead-threads 1 --rc-lookahead 60




I think I may give 2 pass encoding a shot and set my bit rate VBR to 6500 kbps or something.

This way I assume I will get at least a CRF of under 20 on your typical film preset.

Does anyone else have the same dilemma?

Does the encode of example 2 look worse than example 1? If so, you would have to lower --crf until you are happy (you could perhaps use a snippet of the original to save time for testing).
I would assume that example 2 is just a less complex source (e.g. less noisy), and hence the encoder did not waste bits for nothing.
- If your go for a certain file size you need to go for 2-pass, or change --crf in a trial-and-error procedure until you end up with the desired files size. Have fun!
There exists no "encode for a certain quality" as quality this is subjective anyway unless differences are very obvious.

[pcordes]

For example
1.
- Source video only file size 25.3 Gb - Bit rate- 25.6 Mbps
- Encoded file size 6.1 Gb -- Bit rate 6 652 Kbps

2.
- Source video only file size 33.1 Gb - Bit rate- 29.7 Mbps
- Encoded file size 2.7 Gb -- Bit rate 2 590 Kbps

[...]

Does anyone else have the same dilemma?

How is this a dilemma? You didn't say there's anything visually wrong with the 2nd encode, so I'll assume there isn't. Some content is more compressible than others. The bitrate of a crf encode is roughly related (inversely) to how compressible the source is.

Re-encoding the 2nd source with a lower CRF, or 2pass 6Mb/s, would just be throwing way more bits at it than it needs, going way beyond transparent. (Like Sharc said)

The goal of CRF is to have similar quality regardless of the source, not similar bitrate. It's more of a target average QP, I think, which is only approximately related to visual quality. (In ways that depend on the content: some videos will look ok with a higher crf than others.)

But most people just pick a CRF that looks good enough in the worst case, and are happy when some of their encodes come out nice and small because x264 was able to take advantage of a lot of redundancy in the source.

If you want to see how high you can push CRF for a specific source, and have it still look ok, do a test runs at `preset=fast` or something. (NOT ultrafast, me=0 will throw things off). CRF's behaviour is different with different encode settings, but `fast` to `slower` is probably fine for getting close to a crf that will look ok. Encodes at the same CRF usually look fairly similar across a range of presets, with varying bitrate. (This is not a valid way to find the value of settings, because there is quality variation, as well as bitrate variation.)

(edit: does this forum have a way to do an inline version of the [ CODE ] tag? Like backticks on stackoverflow.com.)