Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
|
|
Thread Tools | Search this Thread | Display Modes |
5th March 2020, 00:24 | #1 | Link |
Registered User
Join Date: Dec 2010
Posts: 65
|
Fairly recent, pretty comprehensive, home made x265 3.2 benchmark
I was curious of comparing performances of the latest x265 releases, since I read the article about change into slower/veryslow preset parameters after 3.0 release. I just wanted to see how crf+preset combination could affect compression time and final file size, but never stumbled into a comprehensive benchmark, so I did one my way, using a recent x265 build and a command line as simple as crf+preset (and always "-D 10"). And here you have it too.
Encoding content: y4m file, 1920x1080 2000 frames (~ 6 GB) Source is the "Snowpiercer" bluray movie untouched. Selected footage include action (1000 frames) and more relaxed (not so static, indeed) scenes (1000 frames) for a 2000 frames total. A (resized) sample can be viewed here. Encoder & cpu capabilities: x265 [info]: HEVC encoder version 3.2.1+5-0af31fef64a7 x265 [info]: build info [Linux][GCC 6.3.0][64 bit] 10bit x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 Current kernel: Linux cindy 4.9.0-11-amd64 #1 SMP Debian 4.9.189-3+deb9u2 (2019-11-11) x86_64 GNU/Linux Encoding settings: always 10 bit, preset + crf Encoding duration taken from the x265 final statistics. Code:
| crf 28 | crf 26 | crf 24 | crf 22 | crf 20 | crf 18 | PRESET | (default) | | | | | | --------------------------------------------------------------------------------------- | T: 77.8% | T: 78,5% | T: 85.8% | T: 86.8% | T: 99.1% | T: 103.6% | 4 (fast) | S: 96.5% | S: 126.3% | S: 170.0% | S: 240.0% | S: 357.7% | S: 550.6% | --------------------------------------------------------------------------------------- 5 (medium, | T: 100% | T: 104% | T: 117% | T: 132% | T: 158% | T: 167% | default) | S: 100% | S: 132% | S: 182% | S: 263% | S: 403% | S: 637% | --------------------------------------------------------------------------------------- | T: 341% | T: 361% | T: 403% | T: 444% | T: 528% | T: 625% | 6 (slow) | S: 118% | S: 161% | S: 230% | S: 348% | S: 550% | S: 867% | --------------------------------------------------------------------------------------- | T: 1176% | T: 1251% | T: 1440% | T: 1618% | T: 1927% | T: 2300% | 7 (slower) | S: 126% | S: 174% | S: 251% | S: 380% | S: 590% | S: 907% | --------------------------------------------------------------------------------------- | T: 2211% | T: 2313% | T: 2724% | T: 3051% | T: 3669% | T: 4364% | 8 (veryslow) | S: 125% | S: 172% | S: 248% | S: 376% | S: 586% | S: 903% | --------------------------------------------------------------------------------------- T = encoding time S = raw file size On the other hand, I didn't expect such an increase in file size in vertical direction (same crf value, increasing preset). It seems at lower presets the encoder deliberately discards some data, which is encoded at higher preset values, up to a point where file size remains the same (slower and veryslow). I can figure out more data is encoded more efficiently, so the file size doesn't increase; rather, it is always smaller with veryslow preset, BUT this comes at a cost of doubling the encoding time with respect to the slower preset. |
5th March 2020, 00:24 | #2 | Link |
Registered User
Join Date: Dec 2010
Posts: 65
|
In this second run I encoded the same uncompressed source material (y4m file, 1920x1080 2000 frames, ~ 6 GB) BUT filtered using MCDegrain filter.
Same encoder, same OS, same environment. Code:
| crf 28 | crf 26 | crf 24 | crf 22 | crf 20 | crf 18 | PRESET | (default) | | | | | | --------------------------------------------------------------------------------------- | T: 84.4% | T: 70.3% | T: 73.0% | T: 82.7% | T: 80.6% | T: 87.1% | 4 (fast) | S: 97.7% | S: 122.5% | S: 154.5% | S: 197.8% | S: 258.7% | S: 343.1% | --------------------------------------------------------------------------------------- 5 (medium, | T: 100% | T: 89% | T: 97% | T: 119% | T: 120% | T: 134% | default) | S: 100% | S: 126% | S: 161% | S: 209% | S: 278% | S: 378% | --------------------------------------------------------------------------------------- | T: 353% | T: 313% | T: 332% | T: 457% | T: 403% | T: 425% | 6 (slow) | S: 114% | S: 145% | S: 188% | S: 247% | S: 334% | S: 462% | --------------------------------------------------------------------------------------- | T: 1183% | T: 1067% | T: 1150% | T: 1627% | T: 1414% | T: 1571% | 7 (slower) | S: 116% | S: 149% | S: 192% | S: 254% | S: 344% | S: 476% | --------------------------------------------------------------------------------------- | T: 2077% | T: 1927% | T: 2137% | T: 2943% | T: 2679% | T: 2947% | 8 (veryslow) | S: 115% | S: 147% | S: 191% | S: 252% | S: 342% | S: 473% | --------------------------------------------------------------------------------------- T = encoding time S = raw file size The first thing I notice is the different file size growth, unlike the previous run I didn't have an "explosion" up to 9 times the reference size, and this is due to the filter. Further, there is a weird "jump down" in encoding times between both crf28/crf26 and crf22/crf20, even though visual quality should go up the whole scene compressed easily (but may this only be due to this particular source material?). Also, times are generally shorter: for crf18 encoding times are 20-30% shorter here. In case anyone is wondering, the reference encoding (medium preset, crf28) created a file large 85% than the previous reference, and the encoding took 5% longer. Last edited by Losko; 11th June 2020 at 08:07. Reason: typo |
5th March 2020, 07:16 | #3 | Link |
Pig on the wing
Join Date: Mar 2002
Location: Finland
Posts: 5,731
|
It is an interesting finding compared to x264 is that you get a bigger file when you go to a slower preset. If I remember correctly, in x264 it was said that slower presets generally compress more efficiently. x265 uses more B- and ref frames in slower presets but they don't seem to offset the other things which cause bitrate to rise (probably smoothes the image less so more bits required).
__________________
And if the band you're in starts playing different tunes I'll see you on the dark side of the Moon... |
5th March 2020, 12:29 | #4 | Link | |
Lost my old account :(
Join Date: Jul 2017
Posts: 324
|
Quote:
This behavior has been noted and talked about around here for a long time; slower presets create larger files at the same CRF. It might seem a bit of counterintuitive, but remember that an CRF value does not specify a specific level of quality or fidelity, and its never "constant" between different settings for the encoder. For exempel, try to encode something with fine detail with the faster presets, it will be soft for most if the CRF range, while the slower ones will keep quality at much higher values (and ofc at the same bitrate). It would however be nice if someone could give a more technical explanation of the behavior (more specify maybe what settings that effects it) cause generally with x264 slower presets gave an smaller size then faster ones at the same crf value, the image quality still woudlnt wont be identical though, but the encoder behavior did change in x265. Last edited by excellentswordfight; 5th March 2020 at 13:11. |
|
5th March 2020, 14:19 | #5 | Link |
Registered User
Join Date: May 2006
Posts: 3,997
|
Interesting table. Indeed the strong increase in file size from fast to slow(er) preset for x265 is surprising.
Slower settings producing a larger file size for the same CRF is also possible with x264 however. For example, a stronger (slower) setting for --me and --subme may increase the file size for same CRF, if I remember this correctly. Last edited by Sharc; 5th March 2020 at 14:29. |
5th March 2020, 18:24 | #6 | Link |
Moderator
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,770
|
I think the potential higher size at the same CRF is due to additional rate distortion and psychovisual algorithms being used, which will send more bits to some parts of the image (and less to others, which is why the size gain isn't consistent between sources). However, the file size at the same visual quality goes down as presets go up. For apples-to-apples comparisons, 2-pass VBR to a specified average bitrate is more accurate. Also probably better for benchmarking, as more bits also means more CABAC, so comparing presets at CRF is going to yield slower relative performance at higher presets than would be seen in fixed bitrate or fixed subjective quality. In the real world, no one would encode using the same CRF at different presets, because the whole point of the higher presets is to get more quality per bit.
|
6th March 2020, 11:33 | #8 | Link |
Registered User
Join Date: Dec 2010
Posts: 65
|
Mmmmhhhh, truth is this benchmark wasn't born with quality-wise evaluation in mind but it's a good idea.
I'm currently running an encoding series for another benchmark but I'll see what I can do. Last edited by Losko; 6th March 2020 at 11:42. Reason: typo |
10th March 2020, 22:51 | #10 | Link |
Moderator
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,770
|
VMAF isn't a super good metric to look at subtle differences between CRF18 encodes. And there's no standard way to get an overall VMAF for a clip more than 10-20 seconds long. Harmonic of individual frame scores can work okay if it's just a GOP or two. But for a 10 min clip, the same mean VMAF can indicate a lot of quality variability, or not.
|
8th April 2020, 23:29 | #11 | Link |
Registered User
Join Date: Feb 2002
Posts: 758
|
Question. Imagine you are at Medium CRF 22. Now you want to go one step higher quality. Better to go Slow CRF 22 or Medium CRF 21? Are they the same quality wise? What's the difference?
I think we need a benchmark with a third value. quality if that's possible. Last edited by Atlantis; 8th April 2020 at 23:45. |
9th April 2020, 17:02 | #12 | Link | |
Moderator
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,770
|
Quote:
Sent from my SM-T837V using Tapatalk |
|
10th April 2020, 19:58 | #13 | Link |
Registered User
Join Date: Feb 2002
Posts: 758
|
So I did a test myself.
x265 3.3+10-g08d895bb6 GCC 10.0.1 medium 22 encoded 976 frames in 347.54s (2.81 fps), 5511.89 kb/s, Avg QP:24.58 medium 21 encoded 976 frames in 352.39s (2.77 fps), 6488.14 kb/s, Avg QP:23.57 slow 22 encoded 976 frames in 1174.07s (0.83 fps), 6862.54 kb/s, Avg QP:24.61 As you see preset slow is extremely slower. Uses a lot more bitrate and with my eyes medium 21 looks a lot better than slow 22. Not to mention it's faster and uses less bitrate! What's the point of Slow preset? Why they made it, if you can get better results faster with Medium and lower CRF. Last edited by Atlantis; 11th April 2020 at 01:24. |
11th April 2020, 15:46 | #14 | Link | |
Registered User
Join Date: Jul 2016
Posts: 171
|
Quote:
Same story when comparing Slower (CRF22) vs Slow (CRF21). Personally, after a lot of HDR10 encode comparisons, I decided to use CRF 17 slower for grainy contents and CRF15 slower for everything else. It takes about 35-40h per movie but the quality is great. Last edited by imhh11; 11th April 2020 at 17:02. |
|
11th April 2020, 19:19 | #16 | Link |
Pig on the wing
Join Date: Mar 2002
Location: Finland
Posts: 5,731
|
Downsizing also helps quite a lot.. maybe down to 1440p for sources that are true 4K and 1080p for sources that are actually 2K or so and just upsized to 4K.
__________________
And if the band you're in starts playing different tunes I'll see you on the dark side of the Moon... |
11th April 2020, 22:44 | #17 | Link |
Registered User
Join Date: Jul 2016
Posts: 171
|
Well in my experience with UHD HDR encoding, I often(clean source) get ridiculously low bitrate(10mb/s or less) when I use CRF17+ slower preset.
CRF 15 usually gets me in the 15/20mb/s bitrate which is the size I try to target. |
12th April 2020, 13:37 | #18 | Link | |
Lost my old account :(
Join Date: Jul 2017
Posts: 324
|
Quote:
So did you use filters on your tests above? If so, please always mention this in the future. Its very relevant information when discussing encoder behavior. The point of slower presets is ofc better compression, and that means for a lot of cases, more detail at the same bitrate. Encode something at fixed bitrate (2pass) and compare medium and slower and you will see that slower will keep alot more detail. If you pre-process the material to remove detail, making the source easier to compress the difference could very much be somewhat harder to notice. And if you dont mind the reduction of grain and detail, then good for you, you dont need to wait days for encodes Last edited by excellentswordfight; 12th April 2020 at 13:44. |
|
12th April 2020, 17:54 | #19 | Link | |
Moderator
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,770
|
Quote:
Generally speaking HDR looks better at the same bitrate as the same content in an SDR grade, due to being more perceptually uniform. |
|
12th April 2020, 18:06 | #20 | Link |
Registered User
Join Date: Dec 2005
Location: Germany
Posts: 1,795
|
Hmm so the higher the crf the faster the encoding is. It would be also very interesting to see how ssim / vmaf metrics would look like with your results.
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth VapourSynth Portable FATPACK || VapourSynth Database Last edited by ChaosKing; 12th April 2020 at 18:09. |
Thread Tools | Search this Thread |
Display Modes | |
|
|