Highest quality to file size ratio

[Ischemia24]

I didn't see anything about this in the guides or FAQ and googling also hasn't turned up the specific information I'm looking for, so I'm asking here: How can I use H.265 to achieve the highest quality to file size ratio?

A year or two ago I found this post (https://forum.doom9.org/showthread.php?p=1807501#post1807501) by Ma in the main HEVC thread. Based on that I've been using these Handbrake settings.

1080p
MKV format, 1920x1080, no filters, 10-bit H.265, constant framerate same as source, very slow preset, no encoder tune, auto-profile, 2-pass, 1500 bitrate, Extra Options: rc-lookahead=120:bframes=12:ref=6:subme=7
If it is a dark video then: rc-lookahead=120:bframes=12:ref=6:subme=7:aq-mode=3

720p (only listing what's different here and for subsequent resolutions)
1280x720, 760 bitrate

480p
854x480, 360 bitrate

360p
640x360, 300 bitrate

These are a starting point, and I've had to adjust the bitrate or try CRF instead, depending on the source files. But I've noticed that others have outdone me when it comes to the quality to file size ratio. It seems there should be a better set of options to start with, or I'm missing something in terms of what settings I should know to try to tweak in different scenarios.

I've seen these types of threads before and contributed to them, and they usually have answers like "just find what's right for you", but I would like to at least hear what baseline settings others use and what are the prime candidates for settings to tweak for those that value quality to file size as I do.

[mariush]

My advice would be to seriously analyze how much time disk space is really worth to you.

We're at the point where you can buy a 8 TB (~ 7250 GiB in real disk space) for $150 ... that's 0.02$ per GB.

At 3 mbps, you're looking at ~ 22 MB per minute or 1.4 GB per hour ...

If you're gonna spend maybe 2-3 hours per one hour of encoding you should at least use a higher bitrate, maybe something like 8-10mbps VBR for 1080p and 4-8 mbps VBR for 720p ... if you end up with around 4-6 GB for a 2h movie, you're still looking at less than 0.1$ in disk space cost.

// also hmm.. you joined nov 2016 but today's your first post?

// ps... also checked the other forum post you mentioned.. hope you realized he used a cartoon for tests... real motion video, bluray with grain etc will behave differently.. make sure you understand why he uses some settings there.

[Ischemia24]

My advice would be to seriously analyze how much time disk space is really worth to you.

We're at the point where you can buy a 8 TB (~ 7250 GiB in real disk space) for $150 ... that's 0.02$ per GB.

At 3 mbps, you're looking at ~ 22 MB per minute or 1.4 GB per hour ...

I run an ITX media server, and I don't have a lot of room for expandability. But I recently upgraded it to a 3900X, so I am finally in a position to throw a lot of CPU power at H.265 encoding.

If you're gonna spend maybe 2-3 hours per one hour of encoding you should at least use a higher bitrate, maybe something like 8-10mbps VBR for 1080p and 4-8 mbps VBR for 720p ... if you end up with around 4-6 GB for a 2h movie, you're still looking at less than 0.1$ in disk space cost.

Most of the time the settings I cited in the original post work pretty well for me. But I've seen examples where I'm clearly beaten in quality to file size. I want to know how to improve. Actually, part of that is how to leverage the 3900X, because I started three instances of Handbrake doing 360p and 480p encodes and the CPU is not fully utilized. Maybe should be its own post.

also hmm.. you joined nov 2016 but today's your first post?

ps... also checked the other forum post you mentioned.. hope you realized he used a cartoon for tests... real motion video, bluray with grain etc will behave differently.. make sure you understand why he uses some settings there.

Yes, I was going to ask then what settings I should use for my goals, but I feared I hadn't done enough of my own research and held off, then forgot all about it.

I think the Big Buck Bunny movie sample he used is something that you would tune for Film if using H.264 rather than Animation. But I admit I don't know why he went with the particular settings he chose. It was just presented as "true placebo" mode for H.265 and I seized on it.

[RanmaCanada]

Time to change your media server to something larger. Sorry, but that is the reality. Disk space is cheap, and it will actually cost you more in power to convert your movies, than to just store them. IE the amount it will cost in power to use your computer to "save space" by encoding, is more expensive than storing the original on a hard drive and just buying new drives when you run out of space. Especially if you care about quality in your encodes.

[benwaggoner]

Turning the nebulous concept of "quality" into a linear value that a ratio can be calculated from is a massive unsolved problem in computer vision and human visual perception. Any scale you could come up with would be specific to a given scenario.

[Ischemia24]

I don't feel like the substance of my question is being addressed. I'm not looking to re-encode my entire library. Just a few things, and most new titles I add. I get that quality is difficult to quantify in the context of video encoding, but we do have methods for it, however imperfect.

Any input on encoding settings?

[excellentswordfight]

I don't feel like the substance of my question is being addressed. I'm not looking to re-encode my entire library. Just a few things, and most new titles I add. I get that quality is difficult to quantify in the context of video encoding, but we do have methods for it, however imperfect.

Any input on encoding settings?
Then lets assume that encoding time is pretty much irelevant to the discussion (which sounds crazy to me, but okey). What excactly are you not pleased with? You are talking about examples were you are beaten, how were they beaten? Have you encoded the same content at the same bitrate and compared them? As far as I can see, you are already pretty much throwing everything x265 has to offer at the encode, you can add something like --no-sao and --deblock -1:-1 if you want better detail and noise retention, but other then that, there is not really much more to do, you are already well beyond the diminishing returns point when it comes to compression gains at the cost of cpu cycles.

And for the "Highest quality to file size ratio" part, this is exactly what crf is good for, just start with a baseline crf of 18 or something and increase it until you hit your sweetspot between visual fidelity and size (cause only you can decide at what point you are pleased with quality), at that point you will have found the "best" "quality to fie size ratio". This will also effect your encoding settings, are you looking at maxing out compression with low bitrates and will accept a lot of quality degradation, or should it be close to visually transparent? Cause for example, --no-sao and --deblock -1:-1 is great for the later case, but no that great for the first one.

[Ischemia24]

You are talking about examples were you are beaten, how are you beaten? Have you encoded the same content at the same bitrate and compared them? As far as I can see, you are already pretty much throwing everything x265 has to offer at the encode, you can add something like --no-sao and --deblock -1:-1 if you want better detail and noise retention, but other then that, there is not really much more to do, you are already well beyond the diminishing returns point when it comes to compression gains at the cost of cpu cycles.

And for the "Highest quality to file size ratio" part, this is exactly what crf is good for, just start with a baseline crf of 18 or something and increase it until you hit your sweetspot between visual fidelity and size (cause only you can decide at what point you are pleased with quality), at that point you will have found the "best" "quality to fie size ratio". This will also effect your encoding settings, are you looking at maxing out compression with low bitrates and will accept a lot of quality degradation, or should it be close to visually transparent? Cause for example, --no-sao and --deblock -1:-1 is great for the later case, but no that great for the first one.

I figured I shouldn't be specific, but, it seems common that the release group PSArips put out something that looks better and is smaller than my encodes. And for a specific example, a release group (individual?) who goes by JoyBell released two seasons of Dollhouse at ~1200 kbps 1080p that look better than most of my 1500 kbps 1080p encodes.

I appreciate the input regarding no-sao and deblock -1:-1, thank you.

I can't see the encoding settings PSArips use, but -- actually, I can see the settings on the Dollhouse episodes. Here's one:

Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main 10@L4@High
Codec ID : V_MPEGH/ISO/HEVC
Duration : 50 min 17 s
Bit rate : 1 200 kb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 10 bits
Bits/(Pixel*Frame) : 0.024
Stream size : 432 MiB (81%)
Writing library : x265 2.8+19-bcdc610cf5f0:[Windows][MSVC 1914][64 bit] 10bit
Encoding settings : cpuid=1111039 / frame-threads=3 / numa-pools=8 / wpp / no-pmode / no-pme / no-psnr / ssim / log-level=2 / input-csp=1 / input-res=1920x1080 / interlace=0 / total-frames=72337 / level-idc=0 / high-tier=1 / uhd-bd=0 / ref=3 / no-allow-non-conformance / no-repeat-headers / annexb / no-aud / no-hrd / info / hash=0 / no-temporal-layers / open-gop / min-keyint=23 / keyint=250 / gop-lookahead=0 / bframes=8 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=20 / lookahead-slices=6 / scenecut=40 / radl=0 / no-intra-refresh / ctu=64 / min-cu-size=8 / no-rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=1 / dynamic-rd=4.00 / ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=2 / limit-refs=3 / no-limit-modes / me=1 / subme=2 / merange=57 / temporal-mvp / weightp / weightb / analyze-src-pics / deblock=0:0 / sao / sao-non-deblock / rd=3 / no-early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=0.00 / psy-rdoq=0.00 / no-rd-refine / no-lossless / cbqpoffs=0 / crqpoffs=0 / rc=abr / bitrate=1200 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=2 / cplxblur=20.0 / qblur=0.5 / vbv-maxrate=12000 / vbv-bufsize=24000 / vbv-init=0.9 / ipratio=1.40 / pbratio=1.30 / aq-mode=3 / aq-strength=1.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=0 / overscan=0 / videoformat=5 / range=0 / colorprim=2 / transfer=2 / colormatrix=2 / chromaloc=0 / display-window=0 / max-cll=0,0 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / no-opt-qp-pps / no-opt-ref-list-length-pps / multi-pass-opt-rps / scenecut-bias=0.05 / no-opt-cu-delta-qp / no-aq-motion / no-hdr / no-hdr-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=10 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=0 / limit-sao / ctu-info=0 / no-lowpass-dct / refine-mv-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei
Default : Yes
Forced : No

I don't know what to make of this because they've backed off some of the settings I specify, and I don't know what else they may have specified and what values are defaults. High-tier=1 seems significant, but I'm not seeing anything in the documentation about what benefits that could provide. I'm guessing it means poorer compatibility with decoders, but I'm not concerned about that.

[Forteen88]

VMAF is a better video-quality metric than SSIM & PSNR,
https://streaminglearningcenter.com/blogs/installing-and-using-netflix-vmaf-master.html

[excellentswordfight]

I figured I shouldn't be specific, but, it seems common that the release group PSArips put out something that looks better and is smaller than my encodes. And for a specific example, a release group (individual?) who goes by JoyBell released two seasons of Dollhouse at ~1200 kbps 1080p that look better than most of my 1500 kbps 1080p encodes.

I appreciate the input regarding no-sao and deblock -1:-1, thank you.

I can't see the encoding settings PSArips use, but -- actually, I can see the settings on the Dollhouse episodes. Here's one:

Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main 10@L4@High
Codec ID : V_MPEGH/ISO/HEVC
Duration : 50 min 17 s
Bit rate : 1 200 kb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 10 bits
Bits/(Pixel*Frame) : 0.024
Stream size : 432 MiB (81%)
Writing library : x265 2.8+19-bcdc610cf5f0:[Windows][MSVC 1914][64 bit] 10bit
Encoding settings : cpuid=1111039 / frame-threads=3 / numa-pools=8 / wpp / no-pmode / no-pme / no-psnr / ssim / log-level=2 / input-csp=1 / input-res=1920x1080 / interlace=0 / total-frames=72337 / level-idc=0 / high-tier=1 / uhd-bd=0 / ref=3 / no-allow-non-conformance / no-repeat-headers / annexb / no-aud / no-hrd / info / hash=0 / no-temporal-layers / open-gop / min-keyint=23 / keyint=250 / gop-lookahead=0 / bframes=8 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=20 / lookahead-slices=6 / scenecut=40 / radl=0 / no-intra-refresh / ctu=64 / min-cu-size=8 / no-rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=1 / dynamic-rd=4.00 / ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=2 / limit-refs=3 / no-limit-modes / me=1 / subme=2 / merange=57 / temporal-mvp / weightp / weightb / analyze-src-pics / deblock=0:0 / sao / sao-non-deblock / rd=3 / no-early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=0.00 / psy-rdoq=0.00 / no-rd-refine / no-lossless / cbqpoffs=0 / crqpoffs=0 / rc=abr / bitrate=1200 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=2 / cplxblur=20.0 / qblur=0.5 / vbv-maxrate=12000 / vbv-bufsize=24000 / vbv-init=0.9 / ipratio=1.40 / pbratio=1.30 / aq-mode=3 / aq-strength=1.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=0 / overscan=0 / videoformat=5 / range=0 / colorprim=2 / transfer=2 / colormatrix=2 / chromaloc=0 / display-window=0 / max-cll=0,0 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / no-opt-qp-pps / no-opt-ref-list-length-pps / multi-pass-opt-rps / scenecut-bias=0.05 / no-opt-cu-delta-qp / no-aq-motion / no-hdr / no-hdr-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=10 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=0 / limit-sao / ctu-info=0 / no-lowpass-dct / refine-mv-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei
Default : Yes
Forced : No

I don't know what to make of this because they've backed off some of the settings I specify, and I don't know what else they may have specified and what values are defaults. High-tier=1 seems significant, but I'm not seeing anything in the documentation about what benefits that could provide. I'm guessing it means poorer compatibility with decoders, but I'm not concerned about that.
Are you also encoding dollhouse? If not, it doesnt make much sense to compare another source against it, the compressibility of different sources can be very different, you also dont know if the source were filtered to be able to compress better.

Also, the encode that you have sampled is by no means any special, it looks like something like this was used: --preset medium --profile main10 --bframes 8 --aq-mode 3 --sao-non-deblock --limit-sao --bitrate 1200 --vbv-maxrate 12000 --vbv-bufsize 24000. And afaik, main and high teir only effect allowed datarates @ given level, so it doesnt matter that much here, I would think that it is automatically set cause of --vbv-bufsize 24000.

[Stereodude]

I figured I shouldn't be specific, but, it seems common that the release group PSArips put out something that looks better and is smaller than my encodes. And for a specific example, a release group (individual?) who goes by JoyBell released two seasons of Dollhouse at ~1200 kbps 1080p that look better than most of my 1500 kbps 1080p encodes.
First, lets define "better". What about it makes it visually better? In what regard is it better?

Second, you can see all the settings they used. What's stopping you from trying their settings and experimenting?

[Boulder]

Also, the encode that you have sampled is by no means any special, it looks like something like this was used: --preset medium --profile main10 --bframes 8 --aq-mode 3 --sao-non-deblock --limit-sao --bitrate 1200 --vbv-maxrate 12000 --vbv-bufsize 24000.

It also seems that the psy options are turned off for some reason.

[Ischemia24]

VMAF is a better video-quality metric than SSIM & PSNR,
https://streaminglearningcenter.com/blogs/installing-and-using-netflix-vmaf-master.html

Doesn't look user friendly, but it does look like a good way to evaluate things in a relatively consistent way. I'll see if I can figure it out. Thank you Forteen88.

Are you also encoding dollhouse? If not, it doesnt make much sense to compare another source against it, the compressibility of different sources can be very different, you also dont know if the source were filtered to be able to compress better.

Also, the encode that you have sampled is by no means any special, it looks like something like this was used: --preset medium --profile main10 --bframes 8 --aq-mode 3 --sao-non-deblock --limit-sao --bitrate 1200 --vbv-maxrate 12000 --vbv-bufsize 24000. And afaik, main and high teir only effect allowed datarates @ given level, so it doesnt matter that much here, I would think that it is automatically set cause of --vbv-bufsize 24000.

I admit, I am not also encoding Dollhouse. I've just been using the same settings for a long time and I determined my subjective experience of the videos with the bitrates they had beyond what I could achieve. It's eye opening that the encode settings seem so pedestrian.

Ugh, on further inspection, I do see they kind of "cheated". Filtered to be able to compress better -- I had no idea this was a thing. I see now in some release notes, "The SSIM measures the accuracy of the outputted encode verse the source. It does not reflect on the sharpness or awesomeness of the image, only how close it is to the source material. An "S##" is a score where filters where [sic] not used. "FS##" is an encode where we used filters before the encoder, usually for nasty amounts of grain, this can throw off the score usually in a positive way and we like to be clear and honest [...]". Huh. Looks like the encode settings I copied into this thread were from an episode that had been pre-filtered.

First, lets define "better". What about it makes it visually better? In what regard is it better?

Second, you can see all the settings they used. What's stopping you from trying their settings and experimenting?

Better in this case is subjective, and drawn on all of my past experiences with the encodes I've done. It seemed to me that more detail was retained at a lower bitrate than I've ever managed. The episode[s] looked superior to what I've been able to achieve at that bitrate. What can I say? I give up easily.

I have no idea how this pre-filtering for better compression works, but I'd rather just stick to encoding settings, if I can get Handbrake to utilize the CPU effectively. If pre-filtering has real merit, then I imagine it'll eventually become a part of mainstream encoding software.

It also seems that the psy options are turned off for some reason.

Oh, this is more than one. Psy-rd and Psy-rdoq. That makes experimentation more difficult. I would ask though, do you prefer to use one of these or both? If just one, which one? Or is there a case that merits one over the other, or both?

[Boulder]

Oh, this is more than one. Psy-rd and Psy-rdoq. That makes experimentation more difficult. I would ask though, do you prefer to use one of these or both? If just one, which one? Or is there a case that merits one over the other, or both?

I would leave the psy options as default. They will cause a lower score in those artificial measurement methods, but that's their nature as they are meant to improve subjective quality when you watch the video.

[Stereodude]

I have no idea how this pre-filtering for better compression works, but I'd rather just stick to encoding settings, if I can get Handbrake to utilize the CPU effectively. If pre-filtering has real merit, then I imagine it'll eventually become a part of mainstream encoding software.
Yes, refuse to look into the thing that's most likely responsible for your observations. That's a solid strategy.

[alex1399]

Challenge to the psycho-visual metric school by the non psycho-visual metric theories is not bad thing for beginner. The measure stuffs is important in the context of encoding history.

[benwaggoner]

Doesn't look user friendly, but it does look like a good way to evaluate things in a relatively consistent way. I'll see if I can figure it out. Thank you Forteen88.
VMAF is definitely the least-bad objective metric we've ever had. But it is slow and complex to use, and a lot of decisions need to be made. For example a 720p file encoded from a 720p source will give a higher score than the same file encoded from a 1080p source, since that comparison would scale up 720p to the original frame size.

And how to go from a number for every frame to some kind of title-level score isn't something well defined. A file that bounces from VMAF 50 to 90 is going to be more annoying to viewers than one that's a consistent 70, although both would have the same per-title score. The harmonic mean per GOP or as a 10 second rolling average might be good.



I admit, I am not also encoding Dollhouse. I've just been using the same settings for a long time and I determined my subjective experience of the videos with the bitrates they had beyond what I could achieve. It's eye opening that the encode settings seem so pedestrian.

Ugh, on further inspection, I do see they kind of "cheated". Filtered to be able to compress better -- I had no idea this was a thing. I see now in some release notes, "The SSIM measures the accuracy of the outputted encode verse the source. It does not reflect on the sharpness or awesomeness of the image, only how close it is to the source material. An "S##" is a score where filters where [sic] not used. "FS##" is an encode where we used filters before the encoder, usually for nasty amounts of grain, this can throw off the score usually in a positive way and we like to be clear and honest [...]". Huh. Looks like the encode settings I copied into this thread were from an episode that had been pre-filtered.



Better in this case is subjective, and drawn on all of my past experiences with the encodes I've done. It seemed to me that more detail was retained at a lower bitrate than I've ever managed. The episode[s] looked superior to what I've been able to achieve at that bitrate. What can I say? I give up easily.

I have no idea how this pre-filtering for better compression works, but I'd rather just stick to encoding settings, if I can get Handbrake to utilize the CPU effectively. If pre-filtering has real merit, then I imagine it'll eventually become a part of mainstream encoding software.

Oh, this is more than one. Psy-rd and Psy-rdoq. That makes experimentation more difficult. I would ask though, do you prefer to use one of these or both? If just one, which one? Or is there a case that merits one over the other, or both?

[Ischemia24]

So I have made a few changes to my baseline settings and I feel as though the quality retained at the restrained bitrates I'm using has really leveled up! Thank you Excellentswordfight for the suggestion of using no-sao and deblock -1:-1.

I came across a post claiming that H.264 changes to these settings when tuning for Film (which is what I need most of the time)

deblock=-1:-1
psy-rd=1.0:0.15

So now my default settings look like this.
rc-lookahead=120:bframes=12:ref=6:subme=7:deblock=-1:-1: psy-rd=1.0:0.15:no-sao

I should do some additional testing to find out how I feel about the subjective quality of CRF at values that result in similar bitrates to what I am specifying. I recently had a problematic 480p encode that ended up looking great just changing to CRF 22 (the video bitrate went up ~40% though).

I looked into VMAF and while I appreciate that it exists I decided I would rather subjectively evaluate my encodes.

Are there any other settings I should consider applying to improve the quality to file size ratio? I understand that some settings could be adjusted for better encoding performance in different situations with no noticeable quality loss, and feel free to chime in about that if you'd like, but I would like to get the highest quality to size ratio possible within the scope of the H.265 settings.

How would you approach dealing with a particularly grainy source?

Any thoughts on the usage of the frame-threads setting? A higher number seems like it would increase CPU utilization, but my understanding is that it would go against my goals (and that if I want to be even more ridiculous, I should use frame-threads=1), so I decided I should leave it at the default.

[Boulder]

In a recent enough build, there's the new option --hme which is meant to improve motion estimation.

Personally I've also raised qcomp from 0.6 to 0.7 and set rd=6 and use --rd-refine.

[brumsky]

First let me say, that I used to have a similar goal as you. I used to say the most quality per bit... I'd have encodes that took several days even on a 16c Xeon v4. Then I changed and figured a couple hundred megs more per file wasn't worth days of encoding.

But anyways...

Increase your --bframes to 16 which is the max HEVC supports while saying in spec. You can also use --bframe-bias to increase the likely hood the encoder will use a b frame. I'd say a sane range would be 0-10 for bframe-bias.

You might find --me sea or full will give a bit better compression and quality but with an increase in encoding time.

If you should consider dropping --subme 7 to 5. You'll find --me sea is more likely to provide better quality and possibly a smaller file size.


There is a relatively new setting for motion search --hme. It changes how MS is done. It is supposed to be better quality while being less computational intensive. I've found the settings below are about 7-9% slower while producing a file 1.5% smaller with about the quality as --me 3 --subme 5.

--hme --hme-search 1,2,3

This was about 2.2% smaller but a 43% longer encoding time. I found it noticeably better quality - subjectively of course.

--hme --hme-search 1,3,3



I would suggest trying lower values for --rc-lookahead 120, it is very high. I'm not sure your getting substantially better frame placement vs it set to 60. It'll improve your coding time with negligible difference in size.


You could also set --tu-inter 4 and --tu-intra 4. Those will allow the encoder to look for better compression opportunities.

FYI, I doubt handbrake will support the new HME settings. This was recently made available. You could try something like ripbot264 and download the latest x265 build from the HEVC thread on Doom9.

Good luck.

[Ischemia24]

In a recent enough build, there's the new option --hme which is meant to improve motion estimation.

Personally I've also raised qcomp from 0.6 to 0.7 and set rd=6 and use --rd-refine.

I'm not sure I'm understanding why the qcomp setting is a benefit, other than capitalizing on a high lookahead (not sure what the 'P' in CQP stands for), but I will give these settings a try for sure. Thanks!

First let me say, that I used to have a similar goal as you. I used to say the most quality per bit... I'd have encodes that took several days even on a 16c Xeon v4. Then I changed and figured a couple hundred megs more per file wasn't worth days of encoding.

But anyways...

Increase your --bframes to 16 which is the max HEVC supports while saying in spec. You can also use --bframe-bias to increase the likely hood the encoder will use a b frame. I'd say a sane range would be 0-10 for bframe-bias.

You might find --me sea or full will give a bit better compression and quality but with an increase in encoding time.

If you should consider dropping --subme 7 to 5. You'll find --me sea is more likely to provide better quality and possibly a smaller file size.


There is a relatively new setting for motion search --hme. It changes how MS is done. It is supposed to be better quality while being less computational intensive. I've found the settings below are about 7-9% slower while producing a file 1.5% smaller with about the quality as --me 3 --subme 5.

--hme --hme-search 1,2,3

This was about 2.2% smaller but a 43% longer encoding time. I found it noticeably better quality - subjectively of course.

--hme --hme-search 1,3,3



I would suggest trying lower values for --rc-lookahead 120, it is very high. I'm not sure your getting substantially better frame placement vs it set to 60. It'll improve your coding time with negligible difference in size.


You could also set --tu-inter 4 and --tu-intra 4. Those will allow the encoder to look for better compression opportunities.

FYI, I doubt handbrake will support the new HME settings. This was recently made available. You could try something like ripbot264 and download the latest x265 build from the HEVC thread on Doom9.

Good luck.

Well I do have limits on the amount of time I'll spend. I decided the placebo preset wasn't worth it, and I would rather leave frame-threads at default rather than minimize it. I did spend a solid 7 days encoding Serenity (2005) down to 4.76 GB (4k) on an i5-4430. I thought it looked great, but I think I could reduce the file size with equivalent subjective quality once I become familiar with these newer settings.

I think I will hold off on hme and hme-search until I see it supported in the standard Handbrake download. I would be willing to use the hme-search 1,2,3 setting once that happens though. I'd have to see what I think of hme-search 1,3,3 to decide if I would incorporate that one into my presets.

Taking your word on rc-lookahead=60. I think Ma's post referenced in my OP is probably the only place I've seen it set to 120. The tu-inter and tu-intra settings seem like no brainers, thanks for not letting me miss out on those.

Are there any settings here that would be significantly affected by the output resolution? Like are there settings that become more important as resolution gets higher, or any that are definitely no benefit at 480p or 360p output?

I'll have to search these forums for info on RipBot264. I could make use of the workload distribution its capable of, but the one time I tried to test that it crashed and I found the UI frustrating compared to what I'm used to (Handbrake).

I really appreciate all the helpful info.

[Boulder]

I'm not sure I'm understanding why the qcomp setting is a benefit, other than capitalizing on a high lookahead (not sure what the 'P' in CQP stands for), but I will give these settings a try for sure. Thanks!

A higher qcomp basically means that the encoder will use more bits in the more difficult parts of the frame. This should prevent cases where things get too blurry in high motion etc.

[excellentswordfight]

I'm not sure I'm understanding why the qcomp setting is a benefit, other than capitalizing on a high lookahead (not sure what the 'P' in CQP stands for), but I will give these settings a try for sure. Thanks!

Do note that both the qcomp and psy-rd settings is an double edge sword, the way you are tweaking it now goes against maximum compression and highly compressed encodes that you have discussed previously in this thread. It will give you higher quality output, but at the cost of bigger files, and if you are looking at achieving very low bitrates and will compensate the higher bitrate by increasing the crf value you might end up doing more damage then good with those settings.


Well I do have limits on the amount of time I'll spend. I decided the placebo preset wasn't worth it, and I would rather leave frame-threads at default rather than minimize it. I did spend a solid 7 days encoding Serenity (2005) down to 4.76 GB on an i5-4430. I thought it looked great, but I think I could reduce the file size with equivalent subjective quality once I become familiar with these newer settings.

I think it's a good thing that you dropped the placebo preset, there is a reason why people reacted the way they did in the beginning of the thread. Preset 'slow' is usually a good starting point, it offers a good trade off when it comes to compression/speed.


Are there any settings here that would be significantly affected by the output resolution? Like are there settings that become more important as resolution gets higher, or any that are definitely no benefit at 480p or 360p output?

I wouldnt say that these have no benefit at lower res, but lowering ctu and merange for low res (I guess you can even lower the default values already at 1080p) can have drastic effect on cpu utilization and that way it can also lead to an huge speed increase. I use --ctu 32 --merange 26 for 720p and bellow, which I would say will have an very negligible effect on something like 480p when it comes to quality/compression.

And tbh, I think you are overcomplicating this a bit, the presets are pretty decent in setting appropriate settings based on encoding time trade-offs. And all these settings you are picking up and adding etc, might not be great for your use case, and might not be great for your content, and would need tweaking etc to be appropriate. Its imo better to keep it a bit simpler, at least as a baseline, with the slowest setting you think is realistic for you to use for an real life scenario, then you can look in to settings that affects the specific charactaristics that you are not pleased with.

This is what I personally would use for 720p24 and 1080p24 rec709 SDR material if I would target a bitrate on the lower side (I would guesstimate a bitrate arround 2-3Mbps). I might play arround with the different AQ-modes, and if I were on a 3900x like you I might bump up 'slow' to 'slower'. I would recommend you to use something similar as your baseline to evaluate against.

--preset slow --profile main10 --ctu 32 --merange 26 --crf 22 --keyint 240 --min-keyint 24 --rc-lookahead 48 --bframes 8 --no-sao --deblock -1:-1 --colorprim bt709 --transfer bt709 --colormatrix bt709 --range limited

[brumsky]

Well I do have limits on the amount of time I'll spend. I decided the placebo preset wasn't worth it, and I would rather leave frame-threads at default rather than minimize it. I did spend a solid 7 days encoding Serenity (2005) down to 4.76 GB (4k) on an i5-4430. I thought it looked great, but I think I could reduce the file size with equivalent subjective quality once I become familiar with these newer settings.

I think I will hold off on hme and hme-search until I see it supported in the standard Handbrake download. I would be willing to use the hme-search 1,2,3 setting once that happens though. I'd have to see what I think of hme-search 1,3,3 to decide if I would incorporate that one into my presets.

Taking your word on rc-lookahead=60. I think Ma's post referenced in my OP is probably the only place I've seen it set to 120. The tu-inter and tu-intra settings seem like no brainers, thanks for not letting me miss out on those.

Are there any settings here that would be significantly affected by the output resolution? Like are there settings that become more important as resolution gets higher, or any that are definitely no benefit at 480p or 360p output?

I'll have to search these forums for info on RipBot264. I could make use of the workload distribution its capable of, but the one time I tried to test that it crashed and I found the UI frustrating compared to what I'm used to (Handbrake).

I really appreciate all the helpful info.


I consider Placebo to be the max setting I should use. So like subme 7 I wouldn't go above 5 because placebo doesn't even use it... You'll waste a lot of time for about .1% difference.

As other's have mentioned, you find your encodes will speed up if you drop some settings like these.
ctu32, me-range 26, qg-size 16

However, there is a small chance you could end up with larger files. It's a speed:size trade off.


I'd look into Analysis save and load. You can create a file that saves a lot of the decisions the encoder made during your first attempt at encoding a file. Then if you don't like the outcome and want to make minor changes like bitrate\CRF you can read that log file and it'll take hours to complete instead of days. There is a lot to it so you should read up on it first.

https://x265.readthedocs.io/en/default/cli.html#mode-decision-analysis

[benwaggoner]

I consider Placebo to be the max setting I should use. So like subme 7 I wouldn't go above 5 because placebo doesn't even use it... You'll waste a lot of time for about .1% difference.
The one thing missing from placebo that I've seen be helpful in special cases is --tskip. Not much for natural images, but it can help with very sharp synthetic edges like text, graphics, or CGI-generated cel-style animation.

[Ischemia24]

Do note that both the qcomp and psy-rd settings is an double edge sword, the way you are tweaking it now goes against maximum compression and highly compressed encodes that you have discussed previously in this thread. It will give you higher quality output, but at the cost of bigger files, and if you are looking at achieving very low bitrates and will compensate the higher bitrate by increasing the crf value you might end up doing more damage then good with those settings.

I think it's a good thing that you dropped the placebo preset, there is a reason why people reacted the way they did in the beginning of the thread. Preset 'slow' is usually a good starting point, it offers a good trade off when it comes to compression/speed.

I wouldnt say that these have no benefit at lower res, but lowering ctu and merange for low res (I guess you can even lower the default values already at 1080p) can have drastic effect on cpu utilization and that way it can also lead to an huge speed increase. I use --ctu 32 --merange 26 for 720p and bellow, which I would say will have an very negligible effect on something like 480p when it comes to quality/compression.

And tbh, I think you are overcomplicating this a bit, the presets are pretty decent in setting appropriate settings based on encoding time trade-offs. And all these settings you are picking up and adding etc, might not be great for your use case, and might not be great for your content, and would need tweaking etc to be appropriate. Its imo better to keep it a bit simpler, at least as a baseline, with the slowest setting you think is realistic for you to use for an real life scenario, then you can look in to settings that affects the specific charactaristics that you are not pleased with.

This is what I personally would use for 720p24 and 1080p24 rec709 SDR material if I would target a bitrate on the lower side (I would guesstimate a bitrate arround 2-3Mbps). I might play arround with the different AQ-modes, and if I were on a 3900x like you I might bump up 'slow' to 'slower'. I would recommend you to use something similar as your baseline to evaluate against.

--preset slow --profile main10 --ctu 32 --merange 26 --crf 22 --keyint 240 --min-keyint 24 --rc-lookahead 48 --bframes 8 --no-sao --deblock -1:-1 --colorprim bt709 --transfer bt709 --colormatrix bt709 --range limited

I think I will get a baseline established without qcomp and psy-rd then. I will plan on lowering ctu and merange at least for 360p and 480p. Maybe with some testing I can convince myself to do so for 720p too. Looking again at the presets table (https://x265.readthedocs.io/en/default/presets.html), I think I should start using Slower. But -- why does limit-tu have a value of 4 for the Slower preset, but 0 for every other preset?

My aim for 1080p encodes is pretty much to reach a bitrate as low as ~1200 to 1500 kbps while still looking impressively good to my eyes. 700 to 760 kbps (maybe try pushing down to 650 kbps) for 720p. 340 kbps is often a struggle for some 480p content, and I need to do more testing to determine what range to target for 360p (guessing I can't go much lower than 300 kbps). Basically I want to end up with content that is both impressively small and impressively good looking, but I will sometimes go with "eh, it looks pretty good".

I don't disagree with you that I am overcomplicating it, but I do want to skew things toward maximum compression beyond what most consider a sensible compression/speed balance. Thank you for the suggested baseline. Is there significant efficiency to be gained in specifying the colorspace and limiting the range?

I consider Placebo to be the max setting I should use. So like subme 7 I wouldn't go above 5 because placebo doesn't even use it... You'll waste a lot of time for about .1% difference.

As other's have mentioned, you find your encodes will speed up if you drop some settings like these.
ctu32, me-range 26, qg-size 16

However, there is a small chance you could end up with larger files. It's a speed:size trade off.

I'd look into Analysis save and load. You can create a file that saves a lot of the decisions the encoder made during your first attempt at encoding a file. Then if you don't like the outcome and want to make minor changes like bitrate\CRF you can read that log file and it'll take hours to complete instead of days. There is a lot to it so you should read up on it first.

https://x265.readthedocs.io/en/default/cli.html#mode-decision-analysis

Analysis save and load sounds like it would be worth my time to look into, and I had no idea about it. Thanks.

I'm convinced, I will lower subme to 5. I haven't been specifying qg-size (looks like it was defaulting to 32 in a recent test), so I will go with 16 as a sane value. What is your sense of the time and benefit trade-off if I were to go with 8?

Ctu seems content dependent, and like it could be lowered without efficiency loss for something like a complex computer generated scene where every part of the frame is busy and detailed for a vast majority of frames, but it would be beneficial to keep it at 64 for large numbers of frames with a focused subject and minimal to no detail (like a flat color with no gradient at all) in a large part of the frame. Do I have that right?

I'm thinking for my goals I should probably lower me-range to 26 for 480p and 360p. I'm undecided on 720p and I feel like I should leave it at the default 57 for 1080p. I can see how it could have a significant effect on speed when I'm using me=sea.

The one thing missing from placebo that I've seen be helpful in special cases is --tskip. Not much for natural images, but it can help with very sharp synthetic edges like text, graphics, or CGI-generated cel-style animation.

Sounds like it would come in handy for encoding something like Batman: The Animated Series, Bojack Horseman, Archer, or (maybe?) something like Looney Tunes. I will keep it in mind for an animation focused preset, thank you.

This is great, I've learned more about H.265 settings since starting this thread than I have in the past three years. :D

Looks like I need to compare a CRF 22 encode with the above suggested baseline with what I have as a best attempt for maximum compression settings:
rc-lookahead=60:bframes=16:bframe-bias=5:ref=6:min-keyint=24:me=sea:subme=5:tu-inter=4:tu-intra=4:qg-size=8:deblock=-1:-1:no-sao:colorprim=1:transfer=1:colormatrix=1:range=limited

And I need to determine how I feel about lowering ctu and merange in 720p. I think I'll use the Tears of Steel clip from the challenge thread (https://forum.doom9.org/showthread.php?t=175776).

[Boulder]

In the faster presets than 'Slower', tu-intra and -inter -depths are both 1, so they don't cause excessive calculations. In 'Slower', the depth is changed to 3 and limit-tu 4 is set to restore some of the lost performance.

I think that --rect and --amp could be useful for your use case since they should help with edges. Just set --limit-modes and --limit-refs 3 to fight the resulting slowdown. I also recommend trying the new --hme --hme-search x,y,z instead of --me sea. You can lower merange to 26 if you use CTU 32 and use some other search method than hex. Based on the docs, the value 57 is calculated from CTU 64 (The default is derived from the default CTU size (64) minus the luma interpolation half-length (4) minus maximum subpel distance (2) minus one extra pixel just in case the hex search method is used.)

[brumsky]

The one thing missing from placebo that I've seen be helpful in special cases is --tskip. Not much for natural images, but it can help with very sharp synthetic edges like text, graphics, or CGI-generated cel-style animation.

Nice, I'll have to check it out! Thanks.

What are your thoughts on --rskip? I tend to like it and leave it on...

[brumsky]

When doing 4k encodes I set qg-size to 64. qg-size can help with compression, so I wouldn't go to low with it. You'll find a lot of people recommend ctu 32 qg-size 16 for 1080p content. I usually leave it at 64 and 64. better compression with it, with a very minor loss in quality.

My personal usage would be this.

4k - ctu and qg-size = 64
1080p - cut and gq-size = 64
720p - ctu and qg-size = 32
<720p - ctu = 32, qg-size = 16

To each there own. There are always trade offs...

CTU has to do with the number of pixels the encoder will look at. It can break each block down into smaller pieces based on your settings. --tu-intra and --tu-inter determine how far it can go. The larger the value the further down the tree it can go and the longer the encode can take.

Read this link about how HEVC and encoders work in general. It is a great read!
https://forum.doom9.org/showthread.php?t=167081

rc-lookahead=60:bframes=16:bframe-bias=5:ref=6:min-keyint=24:me=sea:subme=5:tu-inter=4:tu-intra=4:qg-size=8:deblock=-1:-1:no-sao:colorprim=1:transfer=1:colormatrix=1:range=limited

What profile do you use again? Remember qg-size will affect compression levels. I consider 8 to be very small, unless you are doing video at 480p or less.

[Ischemia24]

When doing 4k encodes I set qg-size to 64. qg-size can help with compression, so I wouldn't go to low with it. You'll find a lot of people recommend ctu 32 qg-size 16 for 1080p content. I usually leave it at 64 and 64. better compression with it, with a very minor loss in quality.

My personal usage would be this.

4k - ctu and qg-size = 64
1080p - cut and gq-size = 64
720p - ctu and qg-size = 32
<720p - ctu = 32, qg-size = 16

To each there own. There are always trade offs...

CTU has to do with the number of pixels the encoder will look at. It can break each block down into smaller pieces based on your settings. --tu-intra and --tu-inter determine how far it can go. The larger the value the further down the tree it can go and the longer the encode can take.

Read this link about how HEVC and encoders work in general. It is a great read!
https://forum.doom9.org/showthread.php?t=167081

What profile do you use again? Remember qg-size will affect compression levels. I consider 8 to be very small, unless you are doing video at 480p or less.

I feel like a moron after having read the link. I did not have a comprehension level that would help inform my encoding settings. I will have to go over it a few more times. I always use 10-bit for H.265. I understand that CTUs determine the number of pixels considered at a time, but was I wrong about how I was thinking of CTUs in my previous post as it relates to compression gains to be had with different amounts of the frame with little to no detail? And am I to understand that a qg-size of 8 would result in that setting's highest image quality but lowest compression while a qg-size of 64 would result in that setting's lowest image quality but highest compression? If that is the case then I should probably go with values similar to what you've outlined above.

In the faster presets than 'Slower', tu-intra and -inter -depths are both 1, so they don't cause excessive calculations. In 'Slower', the depth is changed to 3 and limit-tu 4 is set to restore some of the lost performance.

I think that --rect and --amp could be useful for your use case since they should help with edges. Just set --limit-modes and --limit-refs 3 to fight the resulting slowdown. I also recommend trying the new --hme --hme-search x,y,z instead of --me sea. You can lower merange to 26 if you use CTU 32 and use some other search method than hex. Based on the docs, the value 57 is calculated from CTU 64 (The default is derived from the default CTU size (64) minus the luma interpolation half-length (4) minus maximum subpel distance (2) minus one extra pixel just in case the hex search method is used.)

Too much testing to do, I wonder how much performance I'll lose by enabling rect and amp. It's good to know about those though, thank you.

[brumsky]

Generally speaking, yes larger CTUs will be used for areas with little to no motion and static\flat images.

No smaller qg-size don't automatically mean higher quality.

This is taken from the docs.


--qg-size <64|32|16|8>

Enable adaptive quantization for sub-CTUs. This parameter specifies the minimum CU size at which QP can be adjusted, ie. Quantization Group size. Allowed range of values are 64, 32, 16, 8 provided this falls within the inclusive range [maxCUSize, minCUSize]. Default: same as maxCUSize

[Ischemia24]

Generally speaking, yes larger CTUs will be used for areas with little to no motion and static\flat images.

No smaller qg-size don't automatically mean higher quality.

This is taken from the docs.


--qg-size <64|32|16|8>

Enable adaptive quantization for sub-CTUs. This parameter specifies the minimum CU size at which QP can be adjusted, ie. Quantization Group size. Allowed range of values are 64, 32, 16, 8 provided this falls within the inclusive range [maxCUSize, minCUSize]. Default: same as maxCUSize

I looked that option up on the command line options page when I saw it mentioned earlier. I took that to mean that as qg-size gets smaller, there are more calculations/comparisons happening because of more opportunities to make adjustments to improve image quality. It sounds like when qg-size is larger, you're missing out on some potential refinements. Am I incorrectly assuming that the QP adjustments equate to better image quality?

[excellentswordfight]

I think I will get a baseline established without qcomp and psy-rd then. I will plan on lowering ctu and merange at least for 360p and 480p. Maybe with some testing I can convince myself to do so for 720p too. Looking again at the presets table (https://x265.readthedocs.io/en/default/presets.html), I think I should start using Slower. But -- why does limit-tu have a value of 4 for the Slower preset, but 0 for every other preset?

My aim for 1080p encodes is pretty much to reach a bitrate as low as ~1200 to 1500 kbps while still looking impressively good to my eyes. 700 to 760 kbps (maybe try pushing down to 650 kbps) for 720p. 340 kbps is often a struggle for some 480p content, and I need to do more testing to determine what range to target for 360p (guessing I can't go much lower than 300 kbps). Basically I want to end up with content that is both impressively small and impressively good looking, but I will sometimes go with "eh, it looks pretty good".

I don't disagree with you that I am overcomplicating it, but I do want to skew things toward maximum compression beyond what most consider a sensible compression/speed balance. Thank you for the suggested baseline. Is there significant efficiency to be gained in specifying the colorspace and limiting the range?
If you want to go as low as 1200-1500 for 1080p you have think a bit about the settings you are applying, some (most?) people around here when discussing settings are aiming at retaining as much detail as possible, this will most likely not be possible for your case with such a low bitrate target, were you might get better results with a softer image than a sharp one. So for example, turning of SAO (no-sao) increase detail retention, but for those low bitrates the compression gains from SAO might be something you want. To be able to achieve good looking video at such low bitrates, noise and fine detail will have to be sacrificed. This is why the rips you discussed earlier most likely used a denoiser as part of their pre-filtering, making the material easier to compress. x265 and hevc in generell is really good at low bitrates and will soften and blur the image instead of breaking apart, but some of the tuning like deblock, qcomp and psy-rd, and others might not have the effect you are after.

Having that baseline might be a good thing either way, cause then you can make decisions if the aim of a lowering bitrate is worth the trade off in image quality.

No the flags are not there for compression reasons, it's imo best practice to specify them, especially for HEVC since the codec is used for such big range of different color formats.

[Ischemia24]

If you want to go as low as 1200-1500 for 1080p you have think a bit about the settings you are applying, some (most?) people around here when discussing settings are aiming at retaining as much detail as possible, this will most likely not be possible for your case with such a low bitrate target, were you might get better results with a softer image than a sharp one. So for example, turning of SAO (no-sao) increase detail retention, but for those low bitrates the compression gains from SAO might be something you want. To be able to achieve good looking video at such low bitrates, noise and fine detail will have to be sacrificed. This is why the rips you discussed earlier most likely used a denoiser as part of their pre-filtering, making the material easier to compress. x265 and hevc in generell is really good at low bitrates and will soften and blur the image instead of breaking apart, but some of the tuning like deblock, qcomp and psy-rd, and others might not have the effect you are after.

Having that baseline might be a good thing either way, cause then you can make decisions if the aim of a lowering bitrate is worth the trade off in image quality.

No the flags are not there for compression reasons, it's imo best practice to specify them, especially for HEVC since the codec is used for such big range of different color formats.

Well there're two different scenarios, really. In what I've already mentioned, when I aim to reduce the bitrate as far as possible, I want to retain an acceptable level of quality, but the first priority is the lower bitrate. But there are times when I want to emphasize quality over a lower bitrate for some higher profile movies, and that is when I think I should use the constant quality slider, and some settings that do increase quality at the cost of maximum compression like no-sao, deblock=-1:-1, psy-rd (is what is supposedly set when tuning to film, 1.0:0.15, a generally good choice?), and qcomp and rd-refine as Boulder has suggested. So I think I will create presets intended for the constant quality slider which will target perceived quality, and presets intended for targeted bitrates.

Currently testing with a couple high profile movies. I will do a bunch more testing with Tears of Steel as well and submit my CQ and bitrate focused settings for feedback.

[Stereodude]

Then learn how to pre-filter the content since your priority is size with acceptable looking quality, not extreme fidelity to the source.

[RanmaCanada]

You do realize with the amount of time you are wasting attempting this, and the amount of energy, it would still be just far cheaper and easier to build a bigger system than your ITX system. You're eventually going to run out of space, then what will you do? Upgrade your server now, save yourself the hassle, as it's pretty obvious you still have no idea on what you want and what you will find acceptable, as you're going to have to encode movies MULTIPLE times before you get a visual level you deem acceptable as one command line will not work across the board for your desires.

Sorry, but that's what I'm getting from all this. It's futile.