Struggling with my x265 settings

[Hostile_18]

I went to town with all the different AQ, Rskip and no Rect combinations to try and find the sweet spot. Encoding time is in the file names.

In example 1 AQ1 & 2 files are roughly the same. AQ3 modes add roughly + 20mb.
In example 2 AQ1 & 3 files are roughly the same. AQ2 modes add roughly +15mb.


Forrest Gump (1 minute sample)
https://i.ibb.co/mrRnTtwS/Screenshot-2026-02-21-115102.png (https://ibb.co/SD6ky08J)

Beau is Afraid (1 minute sample)
https://i.ibb.co/Gf8kHbLc/Screenshot-2026-02-21-124012.png (https://ibb.co/MytpB34M)

[Z2697]

Not using 10bit?

[microchip8]

I think your CRF of 17 is too low, but that's just me. Keep in mind that the CRF values between x264 and x265 are not the same!. For x264, many have settled on 18 (I use 21) while for x265 it's somewhere between 19 and 21 (I use 20 for hdr and 23 for sdr). For Full HD content, try to stay at or below 10 Mbps. Above that, it's just going to waste bitrate with very diminishing returns wrt quality.

[Hostile_18]

Not using 10bit?

Yeah using 10bit x265.

I think your CRF of 17 is too low, but that's just me. Keep in mind that the CRF values between x264 and x265 are not the same!. For x264, many have settled on 18 (I use 21) while for x265 it's somewhere between 19 and 21 (I use 20 for hdr and 23 for sdr). For Full HD content, try to stay at or below 10 Mbps. Above that, it's just going to waste bitrate with very diminishing returns wrt quality.

Even with CRF17 movies are around 11-13gb, some though go as low as 4gb and 3gb etc... I couldn't rest easy if they went even smaller lol. Im currently hosting full remux files, so it's around a 60% reduction, which I find is decent.

AQ3 with no Rect seems like one of the best in terms of high fidelity plus fast encode times.

[microchip8]

Yeah using 10bit x265.



Even with CRF17 movies are around 11-13gb, some though go as low as 4gb and 3gb etc... I couldn't rest easy if they went even smaller lol. Im currently hosting full remux files, so it's around a 60% reduction, which I find is decent.

AQ3 with no Rect seems like one of the best in terms of high fidelity plus fast encode times.

I find that too big for FHD, but again it's just me. That's a range for x264, not x265 which is better at compression. IMHO, your files should be between 3 and 7GB, depending on content. 11-13GB is something x264-ish

[x264N00b]

AQ3 with no Rect seems like one of the best in terms of high fidelity plus fast encode times.
That's 100% as expected because it uses the same preset with almost the same settings with a ~25% higher bitrate. In case of comparing different settings it's better to set a target bitrate -> 2-pass encoding instead of CRF.
Personally I feel like AQ3 just pushes the overall bitrate, you can also lower CRF about 1.0 or maybe 1,5 and you will get the same result with AQ1. In my tests it never gave me a quality boost in dark scenes =(

A mean VMAF-score of 98 or 99 ist super high, I think 95-96 is about to be transparent quality. But keep in mind it's a average score, you should look for the min VMAF also.

[Z2697]

Yes, the "dark bias" is not directly proportional to the luma, AQ3 bias has a broader range of effect than just improve dark blocks.

[Hostile_18]

Good point about AQ3 adding more bit rate, that's better spent in crf value. AQ seems to be the most consistent, in my admittedly very small sample size of two movies. All the values are pretty similar, but the time saving could be quite substantial, so maybe that is what I will focus on.

[VoodooFX]

Like others said, I think too that CRF 17 is too low for x265, if you need such low CRF then you better use x264.

[Hostile_18]

Theoretically could a lower crf (i.e the one I have) but with faster settings be the best of both worlds? The higher bitrate cancelling out the faster settings? I have a 120TB nas so its not essential that the file is in the smallest compression, just that its good enough savings, that it is worth doing.

I haven't considered average bit rate too much, perhaps that's an option, or is CRF the better choice for quality?

EDIT: x264 would be alot faster, but I have no idea the settings I'd use and don't want to spend 20+ hours of experimentation lol.

[VoodooFX]

Usually, at such high bitrates [CRF 17], x264 gives better subjective quality and it's much faster.
IMO, if you want to use CRF for x265 lower that ~20-21, then go with x264.

x264 settings are basically similar to x265 [minus x265 overcomplication].

[Hostile_18]

Usually, at such high bitrates [CRF 17], x264 gives better subjective quality and it's much faster.
IMO, if you want to use CRF for x265 lower that ~20-21, then go with x264.

x264 settings are basically similar to x265 [minus x265 overcomplication].

x264 is super fast on my CPU, but the blur (likely the AQ) on the edges is terrible, really bad.

[VoodooFX]

x264 is super fast on my CPU, but the blur (likely the AQ) on the edges is terrible, really bad.

Do you mean the issue when black borders are not cropped? Just crop them.

[Hostile_18]

Do you mean the issue when black borders are not cropped? Just crop them.

I can't because then it changes the PGS subs, which are location based.

[Z2697]

I use x265 at CRF 14 quite a lot, LOL

[VoodooFX]

I can't because then it changes the PGS subs, which are location based.

I think there is some workaround adding "virtual" black borders. Maybe some mkv flags?

[GeoffreyA]

Theoretically could a lower crf (i.e the one I have) but with faster settings be the best of both worlds? The higher bitrate cancelling out the faster settings? I have a 120TB nas so its not essential that the file is in the smallest compression, just that its good enough savings, that it is worth doing.

I haven't considered average bit rate too much, perhaps that's an option, or is CRF the better choice for quality?

EDIT: x264 would be alot faster, but I have no idea the settings I'd use and don't want to spend 20+ hours of experimentation lol.

Bitrate is king, so yes, if there are enough bits, it will outweigh preset speed. However, for archival-grade encodings, you'll not want to use faster presets.

Stick with CRF for quality. Use two passes to target a certain bitrate or file size, the tradeoff being that, if the bitrate isn't big enough, quality will vary from film to film.

For x264, try:

-level 4.1 -preset veryslow -tune film -crf 18/21 -aq-mode 3

If acceptable from size, quality, and speed points of view, the settings can be further tuned.

[microchip8]

Om my overclocked 4 GHz Ryzen 9 5900X, I've maxed out all possible settings and it's still 2x faster than realtime encoding for FHD. This is 'cause H.264/x264 is not that demanding and complex like HEVC/x265 is. The latter was a big upgrade over x264 with a lot of computational-heavy options. It's possible that in 5-10 years from now (ruff estimate), you'd see the same for x265 as is now the case with x264.

I'd take 10-bit x264 over 10-bit x265 most of the time. The problem is compatibility. 10-bit H.264 is virtually unsupported by most platforms/devices

[GeoffreyA]

Yes, with x264 on contemporary computers, there's no need to adjust for speed.

[Hostile_18]

AV1 is really fast on my 9800x3D as well, very close to x264.

At the moment I have two file flow settings ready to apply to my library. One just cleans up files (Removes all but main audio and commentary tracks, all non English subtitles, all embedded metadata, renames remaining tracks in a standardized format, sets forced subtitle if present as default etc). This takes about 10 minutes per movie.

The other option is similar but encodes video and audio, largely with the settings I posted a while back. This takes about 2.5 hours a movie, though occasionally one will flag that takes 4 plus hours. I'd love to encode in x264 but the border issues make it a total no go. On my main screen it can be a cm top and bottom of distortion, with a grainy film, if borders are not cropped.

[microchip8]

Have you tried lossless encoding with x265 as I suggested a few posts back?

If you're that concerned with tiny imperfections in lossy encoding and nitpick a lot about how grain/noise looks, try lossless encoding. From what I gathered, you're not that concerned about file size much...

[Hostile_18]

Have you tried lossless encoding with x265 as I suggested a few posts back?

If you're that concerned with tiny imperfections in lossy encoding and nitpick a lot about how grain/noise looks, try lossless encoding. From what I gathered, you're not that concerned about file size much...

I am not familiar with lossless encoding. My understanding was that anything that could be encoded is lossy, and that past a certain point, you get bigger file sizes than source and still not as accurate?

[microchip8]

I am not familiar with lossless encoding. My understanding was that anything that could be encoded is lossy, and that past a certain point, you get bigger file sizes than source and still not as accurate?

It works similarly like in audio, eg FLAC is lossless. In other words, it compresses frequently repeating data only that can be reconstructed perfectly upon decoding. So upon decoding, you get a perfect 1:1 output as if you're looking at the original. It usually makes the output by maybe ~10 or so percent smaller.

As opposed to lossy encoding where it discards data and poef, it's forever gone and can't be reconstructed upon decoding.

[benwaggoner]

AV1 is really fast on my 9800x3D as well, very close to x264.
With which encoder and preset??? AV1 is typically more like 10x the speed of x264. And AV1 has more fragile psychovisual tuning if you want material bitrate savings (although is certainly is improving year-on-year).[/QUOTE]

[benwaggoner]

It works similarly like in audio, eg FLAC is lossless. In other words, it compresses frequently repeating data only that can be reconstructed perfectly upon decoding. So upon decoding, you get a perfect 1:1 output as if you're looking at the original. It usually makes the output by maybe ~10 or so percent smaller.
For video content you can often get >50% reduction with a well-tuned lossy encode, depending on how much noise there is. Something like grain-free CGI can be shrunk a lot more.

[GeoffreyA]

With which encoder and preset??? AV1 is typically more like 10x the speed of x264. And AV1 has more fragile psychovisual tuning if you want material bitrate savings (although is certainly is improving year-on-year).

SVT-AV1 can be pretty fast, depending on the preset. So for a certain level of speed, it might well achieve better compression than x264.

[RanmaCanada]

SVT-AV1 can be pretty fast, depending on the preset. So for a certain level of speed, it might well achieve better compression than x264.

Sure, if you like a blurry mess. It's sadly still a major work in progress and I can get much better results with x265 currently than I can with AV1. AV1 has it's place, but it's still not ready for primetime as there is still far too much branching and side projects that keep getting abandoned, or they get merged and all your switches change and values invert.

AV1 is a mess.

[GeoffreyA]

Sure, if you like a blurry mess. It's sadly still a major work in progress and I can get much better results with x265 currently than I can with AV1. AV1 has it's place, but it's still not ready for primetime as there is still far too much branching and side projects that keep getting abandoned, or they get merged and all your switches change and values invert.

AV1 is a mess.

No doubt, x264 and x265 are the gold standard in quality, and SVT-AV1 still suffers from loss of detail and, worse, temporal strobing on grainy content. But from a pure compression-per-speed point of view, it can beat x264 and retain serviceable quality.

Intel's hardware implementation of AV1 seems to be roughly on par with SVT-AV1, bringing further speed.

[excellentswordfight]

A few days ago, I read an article that the upcoming Intel Nova Lake desktop CPUs can draw up to 700 Watts!!! under full load! Jesus! Intel has totally lost it and has surpassed the power hungry Pentium 4 joke of many, many moons ago.

I would definitely NOT consider an Intel system for encoding. I'm on Linux, but their hybrid architecture still gives problems on Windows that the majority of people use.

What is the point of this hybrid E, LP and P cores? It's fine if the E/LP cores can significantly reduce power draw at idle, but once you start using the P cores for an extended period like during encoding, you might get a big surprise on your next power bill! :)

Yeah, AMD it is for the (near) future!
O.T
Pretty sure those 700W was for PL4 (which we have to wait and see what that will mean in practice), I dont think much can be said about the efficiency based on that, PL1 was still 150W.

But what we do know, is that Phanter Lake is the most efficient x86 design available, which is good indicator that Nova Lake looks quite promising, as this is the first time in a long time were Intel has passed AMD in efficiency, especially efficiency for maximum compute loads (Intel has always been good at efficiency at lower power states). Its also a prmocessing indicator that Intel is back on track with their manufacturing, as Phanter Lake is back (for the compute die) on their own process node.

https://phoronix.com/benchmark/result/intel-core-ultra-x7-358h-linux-cpu-performance/x265-bosphorus-4k-2.svgz

[microchip8]

For video content you can often get >50% reduction with a well-tuned lossy encode, depending on how much noise there is. Something like grain-free CGI can be shrunk a lot more.

Indeed, but I mentioned lossless because he wasn't happy with the lossy results, nitpicking this or that doesn't look as good as the source...

[Hostile_18]

Indeed, but I mentioned lossless because he wasn't happy with the lossy results, nitpicking this or that doesn't look as good as the source...

I wouldn't say nit picking, just testing and refining before been applied to the whole library. It makes sense to do all that first, before rolling it out on a larger scale.

[microchip8]

I wouldn't say nit picking, just testing and refining before been applied to the whole library. It makes sense to do all that first, before rolling it out on a larger scale.

Sure, but at some point, with lossy compression, you have to realize that you'll get imperfections. And you'll come across content that doesn't look as good as you'd expect while others do. I don't think there's a magical formula that works on 100% of all content.

[Hostile_18]

Sure, but at some point, with lossy compression, you have to realize that you'll get imperfections. And you'll come across content that doesn't look as good as you'd expect while others do. I don't think there's a magical formula that works on 100% of all content.

Yeah that is fair. I've come along way since my OP post, but there probably isn't much more extra quality to extract.

Grain is the enemy of all mortal men, especially if you aim to preserve it to hold the detail.

[benwaggoner]

Yeah that is fair. I've come along way since my OP post, but there probably isn't much more extra quality to extract.

Grain is the enemy of all mortal men, especially if you aim to preserve it to hold the detail.
QFT!

At least until we can get a good grain synthesis technology in a codec. AV1's was a good start, and AVFG1 is better yet, although still has limitations from a maximum 64x64 synthesized grain map used for everything.

[microchip8]

Yes, grain is a very evil thing, especially when it's added artificially. I don't get why some mastering people do this. Along with judder which I hate with a passion (do you see juddery when looking at the world? No? Why should it be different for video?) grain is my second enemy!

[rwill]

Grain is an artistic style element and you can encode it just fine, it just takes some trial and error when starting out. No need to hate on it just because you cannot encode it well yet.

[Z2697]

Can't agree with the obviously fake grain in some modern movies.
Real / legit "grains" are more acceptable, whether it's film grain or "digital grain" (noises from sensor).

[jpsdr]

Grain and/or noise, but i'll talk more about grain, is very relative to people.
For me, i don't like it. And even more on old movies, when film deteriorate with age, making them look worse (and sometimes a loooooot) than when they were released decades ago.
So, there is people who are not bothered with it, or even more like it, and there is people who don't... :D
This i why, on my personnal encodes, i remove grain on old film movies.

[microchip8]

Grain is an artistic style element and you can encode it just fine, it just takes some trial and error when starting out. No need to hate on it just because you cannot encode it well yet.

I do not hate it because I can't encode it - I have main grainy movies. I hate it because when I look at the world, I do not see it grainy. You do? Why should it be any different in video? Artistic or not, I do not like/want grain

[GeoffreyA]

For my part, I like grain, but it's a curse to encode. Artificial grain is less defensible, like artificial flooring made to look like wood.

Dune, 2021, is an interesting case: shot digitally, it was printed onto film and scanned back, giving a fine, grainy texture. From what I can see, the scenes on rainy Caladan are smooth, but as soon as the setting moves to Arrakis, the desert, the grain kicks in, usually fine, but thick during the visions—again, fitting to the context.

[microchip8]

Regarding judder, I read a survey that most people prefer it. Of course they do! They've been babyfed and brainwashed by Hollywood and TV in general that judder is good and is part of the "cinematic" experience. I strongly disagree with that. It gives me a headache and "eye problems" when a movie is juddery. The biggest judder I've seen is in Star Wars: Attack Of the Clones, where Obi Wan walks around the clone cocoons with the two Camino aliens. It is totally unwatchable, ugly and "shocking" when the camera pans. Luckily my TV has motion smoothing that gets rid of most judder but it still needs improvements.

Grains I can tolerate a bit if it's not that strong/in your face. I sometimes add a bit of noise to very clean encodings to not introduce banding, but this noise is not visible from a distance like thick grain. Besides wasting bits and slowing down the encoding, I do not see any benefits in grain. It does not look more "authentic" than a clean picture.

[GeoffreyA]

Judder is indeed tortuous to watch, and a major defect. By the same token, I don't like high-fps video, which makes my head spin. Proper 23.976 fps, without judder, looks pleasing.

[x264N00b]

Worst judder I've seen is in Pulp Fiction when Travolta walks through Jack Rabbit Slims and the camera , right after the "all right", moves from the posters to the singer. That must have something to do with the camera shutter speed, right?

[Hostile_18]

I'm not a fan of grain, but I don't go out my way to remove it as it nearly always takes away detail with it. I don't like that they add in grain on modern movies, there's lots of different artistic ways to set mood and ambience without putting a noise filter over it. Some games often have this fake grain filter also (along with depth of field, chromatic aberration etc). It seems like light grain films encode fine, heavy grain films are fine, but it does seem like everyone's settings struggled on the Forrest Gump remaster in terms of maintaining 1:1 grain structure.

I tweaked my settings (mainly the maxrate and bufsize) and have done about 40 movies now. Average reduction is 33% of source, which I am happy with. I just focused on high quality steaming versions that are typically encoded in real time. I decided on 5.1 sound 640 Opus as that had a maximum compatibility with my client devices (increasing the bit rate past >700kbps caused issues, without work around). If a movie really needs it the max limit is 7.5 gb an hour.

libx265 -pix_fmt yuv420p10le -crf 17 -preset slow -x265-params aq-mode=1:no-open-gop=1:high-tier=1:level-idc=4.1:bframes=6:subme=4:rc-lookahead=120:pbratio=1.25:ctu=32:merange=26:deblock=-3,-3:no-sao=1:vbv-maxrate=16000:vbv-bufsize=64000:vbv-init=0.9:rskip=2:rskip-edge-threshold=1

[benwaggoner]

Grain and/or noise, but i'll talk more about grain, is very relative to people.
For me, i don't like it. And even more on old movies, when film deteriorate with age, making them look worse (and sometimes a loooooot) than when they were released decades ago.
This is actually a serious issue. Older movies were meant to be projected against a perforated high gain screen from a 35mm film projector that vibrated and projected through glass, at 14 foot-lamberts (and typically less in most theaters).

An 8K film scan of the negative will have ENORMOUSLY more grain than anyone ever saw in a movie theater until the last 20 years, including the creatives. And if you remaster for HDR, that grain can wind up getting color corrected in ways that real-world grain never could have been.

It isn't "creative intent" to include much more sharp fine grain than the creators ever saw when making the film, or after its release. And it is a failure of creative intent to leave it there, in my (strong and correct) opinion.

[benwaggoner]

Worst judder I've seen is in Pulp Fiction when Travolta walks through Jack Rabbit Slims and the camera , right after the "all right", moves from the posters to the singer. That must have something to do with the camera shutter speed, right?
Shutter speed has some impact on judder; slower shutters mean more motion blur which mediates judder some. But the default is already a 180 degree shutter; 1/48th of a second exposure at 24 fps.

The real problem is that 24 fps was chosen because it was the lowest frame rate where lip sync didn't look artificial, in an era of heavy fixed position cameras and dollies. Handling fast motion wasn't a design goal, and it's not good at it. Hence the "seven second rule" and all the other cinematographic tricks to have things seem to be moving fast while not introducing judder. Like having a close up on a moving item and tracking the item while the fast moving background is outside the depth of field and thus blurry. Look how basketball in a movie is shot versus watching a game on TV. Movies have all these close ups of hands and balls going into a fixed basket and such, because an actual basketball game looks terrible and is hard to even see what is happening at 24p.

The Nyquist limit is the fundamental issue. At 24 fps, any motion that happens or changes in 1/12th of a second or less gets lost. Basketballs teleport instead of dribble or fly. Train wheels stop moving then start turning backwards as they pass 12 rpm.

[excellentswordfight]

Shutter speed has some impact on judder; slower shutters mean more motion blur which mediates judder some. But the default is already a 180 degree shutter; 1/48th of a second exposure at 24 fps.

The real problem is that 24 fps was chosen because it was the lowest frame rate where lip sync didn't look artificial, in an era of heavy fixed position cameras and dollies. Handling fast motion wasn't a design goal, and it's not good at it. Hence the "seven second rule" and all the other cinematographic tricks to have things seem to be moving fast while not introducing judder. Like having a close up on a moving item and tracking the item while the fast moving background is outside the depth of field and thus blurry. Look how basketball in a movie is shot versus watching a game on TV. Movies have all these close ups of hands and balls going into a fixed basket and such, because an actual basketball game looks terrible and is hard to even see what is happening at 24p.

The Nyquist limit is the fundamental issue. At 24 fps, any motion that happens or changes in 1/12th of a second or less gets lost. Basketballs teleport instead of dribble or fly. Train wheels stop moving then start turning backwards as they pass 12 rpm.
It should also be noted that your TV will impact this quite a bit as well, especially in recent years. LCD:s/plasma has historically been quite slow, adding quite a bit of motionblur themself, smoothing over issues created low framerates. So modern pannels with faster pixels, and especially Oleds are way less forgiving when things like the the 7-second rule is broken, which can lead to stutter, even if they displaychain is setup in the correct way to display 23.976/24p content.

[benwaggoner]

It should also be noted that your TV will impact this quite a bit as well, especially in recent years. LCD:s/plasma has historically been quite slow, adding quite a bit of motionblur themself, smoothing over issues created low framerates. So modern pannels with faster pixels, and especially Oleds are way less forgiving when things like the the 7-second rule is broken, which can lead to stutter, even if they displaychain is setup in the correct way to display 23.976/24p content.
Indeed. Honestly, the more I learn about display panels and tone mapping and the whole display chain, the more it feels like some kind of dark alchemy. It only seemed straightforward before I was paying deep attention.

[Hostile_18]

I upgraded my server to a Ultra i7 270k Plus for $300. It beats out my 9800x3d in my gaming rig at encoding x265 by a massive 40%. Plus I wont have to have both my server and gaming rig on at the same time... although actually maybe I cohld have them both converting. Decisions.

I have a choice now between much faster than real time encoding or tweaking my settings to be even higher quality at the same speed. :D

[GeoffreyA]

The 270K and 250K have been getting good reviews and are disruptive for the price, particularly the latter. AMD needs to wake up, having grown greedy at the top.