New To x264 - Coming From x265 - High Detail - Bframe Quality and more...

[HD MOVIE SOURCE]

Hi,

I'm new to x264, and I'm coming from x265. I'm just wondering if any of the settings from x265 have any equivalence to x264?

The first thing I noticed was that x264 looks over-sharpened and edgy. So, I found that no-psy gives a similar look to x265, but even then there is some edge. With that said are there any settings in x264 that control detail outside of psy controls? I've read that the deadzones can control fine detail. I'm after as much detail as the encoder will allow.
Is deadzone-inter=0 and deadzone-intra=0 absolute maximum detail?
Are there any other settings that specifically go after detail?

Like x265, I keep to uhd-bd=1 just to give me a sense of a spec to go after. With x264 I'm using bluray-compat=1. I do this to learn how to maximize encodes for a particular constraint. I enjoy doing this. Are there any restrictions using bluray-compat=1?
I know that Blu-ray can only use 3 bframes, which is honestly fine. Also it can only use as high as b-pyramid=1, which I've found to be a quality loss anyway, so I just use b-pyramid=0.

Are there any other specific settings that are required for Blu-ray compatibility? How many refs is it 3 or 4?

With x265 use can use ctu=64, would an equivalent setting in x264 be 8x8dct=1? Is that as high in blocks as we can go?

I read here: http://www.chaneru.com/Roku/HLS/X264_Settings.htm that slices on Blu-ray should be set to 4, is this accurate?

Are lookahead_threads=0 the same as x265? I typically set those to zero.


Another thing I'd like to know, exactly which settings increase bframe and pframe quality? Using x265 we have b-adapt=2, b-intra, b-pyramid, weightb, weightp.
I've seen that x264 has some similar options, but it does haven't b-intra, are there any more settings where I can get even more quality out of bframes?


Thanks for reading, any info is welcome. I'm basically trying to get as much detail without that artificial overshrpened look, and keeping to a restriction like the Blu-ray format because it helps me try to maximize what I have. I prefer to learn that way rather than just maxing everything out.

[microchip8]

x264 produces better fine details, even without tweaks - it's the other way around. People coming from x264 often complain that x265 looks too soft. You have to tune the hell out of x265 to come close to the x264 detail preservation. That said, If you tweak x264 too much, it'll look over-sharpen. I suggest go with the defaults of a specific high-quality preset like 'slow' or 'veryslow'. The only thing that's overkill in these presets is the ref count. For live-action content, anything more than 6 can degrade quality. For anime, up-to 16 can be useful. The only thing I'd do is lower deblock a bit (-1,-1 is fine) and disable decimate with --no-dct-decimate. x264 has in addition to presets, also psy tunings. Might try them.

Regarding Blu-ray, the only other thing you missed is lower amount of keyint and a closed GOP.

[HD MOVIE SOURCE]

Originally Posted by microchip8

x264 produces better fine details, even without tweaks - it's the other way around. People coming from x264 often complain that x265 looks too soft. You have to tune the hell out of x265 to come close to the x264 detail preservation. That said, If you tweak x264 too much, it'll look over-sharpen. I suggest go with the defaults of a specific high-quality preset like 'slow' or 'veryslow'. The only thing that's overkill in these presets is the ref count. For live-action content, anything more than 6 can degrade quality. For anime, up-to 16 can be useful. The only thing I'd do is lower deblock a bit (-1,-1 is fine) and disable decimate with --no-dct-decimate. x264 has in addition to presets, also psy tunings. Might try them.

Regarding Blu-ray, the only other thing you missed is lower amount of keyint and a closed GOP.

This is true, I spent a long time trying to get x265 from being so soft to having as much detail as possible. It takes a lot to get there. I noticed that x264 had too much detail though, and maybe in an artificial way. It almost looked like edge enhancement, but maybe there's a fine balance there with the psy adjustments. I think it will only need small number for psy. I will have to do some more testing.

Right now I'm using no-deblock with CRF=0, with a bandwidth restriction of vbv-maxrate=40000 and vbv-bufsize=40000. I don't see any artifacts of blocking of any kind. Does x264 suffer from image softening when using deblock? This is one reason I didn't want to use it. It appears at high bit-rates that its not needed as much.

[excellentswordfight]

Originally Posted by HD MOVIE SOURCE

Are there any other specific settings that are required for Blu-ray compatibility? How many refs is it 3 or 4?

Its 4 for 1080p (it will be limited for you when setting bluray-compat),, you can find find the detailed speccs here (see annex a):

https://web.archive.org/web/20151028...per_110712.pdf

There is also a good writeup of the important parameters here:

https://forum.doom9.org/showthread.php?t=154533

In general, this is what you want to set for 1080p24:

--profile high --level 4.1 --bluray-compat --vbv-maxrate 40000 --vbv-bufsize 30000 --keyint 24 --slices 4

Open-GOP is also allowed according to the specs, so you can also use --open-gop, its usually rather beneficial at small GOP-sizes (which the specc mandates). Also note that the bufsize limit of bluray is 30Mbps, not 40.

Use that with --preset veryslow & --tune film and nothing else with something like 2pass vbr @ the mid to high 20s, as starting point, (and if you dont wanna use it as a starting point, atleast use it as reference point to compare your encoding to). Analyze the produced image and start to evaluate what you are not happy with/what issues you are having and start tuning based on that. This should not produce a oversharpened look, or soft look, using deblocking strength from -1 to -3 is rather common with x264 and should be a rather decent balance for most live-action content (the tune film preset uses -1).

And please always post your complete command line, otherwise its close to impossible to give feedback. But from the sound if its already rather unorthadox so it might be hard to give advice as the encoder will operate in a very different way to what most people are used to ("no-deblock with CRF=0, with a bandwidth restriction of vbv-maxrate=40000 and vbv-bufsize=40000").

[HD MOVIE SOURCE]

Originally Posted by excellentswordfight

Its 4 for 1080p (it will be limited for you when setting bluray-compat),, you can find find the detailed speccs here (see annex a):

https://web.archive.org/web/20151028...per_110712.pdf

There is also a good writeup of the important parameters here:

https://forum.doom9.org/showthread.php?t=154533

In general, this is what you want to set for 1080p24:

--profile high --level 4.1 --bluray-compat --vbv-maxrate 40000 --vbv-bufsize 30000 --keyint 24 --slices 4

Open-GOP is also allowed according to the specs, so you can also use --open-gop, its usually rather beneficial at small GOP-sizes (which the specc mandates). Also note that the bufsize limit of bluray is 30Mbps, not 40.

Use that with --preset veryslow & --tune film and nothing else with something like 2pass vbr @ the mid to high 20s, as starting point, (and if you dont wanna use it as a starting point, atleast use it as reference point to compare your encoding to). Analyze the produced image and start to evaluate what you are not happy with/what issues you are having and start tuning based on that. This should not produce a oversharpened look, or soft look, using deblocking strength from -1 to -3 is rather common with x264 and should be a rather decent balance for most live-action content (the tune film preset uses -1).

And please always post your complete command line, otherwise its close to impossible to give feedback. But from the sound if its already rather unorthadox so it might be hard to give advice as the encoder will operate in a very different way to what most people are used to ("no-deblock with CRF=0, with a bandwidth restriction of vbv-maxrate=40000 and vbv-bufsize=40000").

Thanks for the feedback and the links, Ive been looking for things like this but came up with nothing. I really appreciate it.

If you use a buffsize of 30 Mbps and a bit-rate of 40 Mbps, would that create underflow?

My current settings are CRF=1 because 0 causes x264 to think its in lossless mode and then disables it, so I just use CRF=1 and the QP values were equal to CRF=0 anyway.

My current settings are being tested right now, but I' also done. I use very slow because it gives lower QP values than placebo for some reason if using the same settings.

This is the highest quality Ive been able to get so far using Vidcoder.

bluray-compat=1:no-open-gop:min-keyint=1:keyint=24:rc-lookahead=24:slices=4:scenecut=40:vbv-maxrate=40000:vbv-bufsize=40000: pbratio=1.00:ipratio=1.00:ref=3:constrained-intra=0:mixed-refs=1:bframes=3:b-adapt=2:b-pyramid=0:weightp=1:weightb=1:no-deblock:subme=11: psy=0:mbtree=0:deadzone-inter=0:deadzone-intra=0:no-dct-decimate:cabac=1:8x8dct=1:direct=auto:analyse=all:me=tesa:merange=24:chroma-me=0:trellis=2:fast-pskip=0: partitions=all:aq-mode=3:aq-strength=1.7

An example of the frame quality on a prores trailer I'm getting so far and I haven't finished optimizing quite yet.
HB x264 [info]: frame I:181 Avg QP: 7.99 size:303143
HB x264 [info]: frame P:1259 Avg QP: 8.94 size:211721
HB x264 [info]: frame B:2224 Avg QP: 8.76 size:159667

It's interesting how chroma-me being turned off increased quality by a little. And after testing all the AQ modes, aq-mode 3 yielded the lowest QP, and I'm currently trying to find the best setting there. On x265, on grainless content, it was 1.7, so I'm currently testing that. b-pyramid I actually lose quality, which I found odd because it's a quality increase on x265. I've run tests with 1, 2 and 3 bframes, and all can look very good. But using b-adapt allows better control I think. I tried no-scenecut and I lost quality by quite a bit. The reason I tried that is that trailers have lots of scenecuts, but without a good reference frame, x264 seems to lose quality quite quickly. It needs that iframe, otherwise, the p and bframes lose quality. x265 seems to be able to get away with many more bframes without losing quality which is really nice, but requires so much CPU power.


I will test a buffsize of 30'000, hopefully, it won't pull the quality down too much. I've never tried using an open GOP on x265 or x264. It still requires a keyint of 24 though right? If so I don't see how an open GOP would change things though, what does it offer if the keyint must be 24?

Thanks.

[excellentswordfight]

Originally Posted by HD MOVIE SOURCE

Thanks for the feedback and the links, Ive been looking for things like this but came up with nothing. I really appreciate it.

If you use a buffsize of 30 Mbps and a bit-rate of 40 Mbps, would that create underflow?

I think you are meaning overflow? But the maxrate is the max-rate, it doesnt mean that you will constantly feed the buffer at that speed (the maxrate for bluray is based on the read-speed of the phyiscal discs, hence the lower value than the h264 spec allows), when both are set the encoder will make sure it doesnt over/underflow on decoders fit for those values. I never really understood why the bluray-spec dictates a lower bufsize, my best guest would that at the time it was set cheap decoders had a rather small buffer.

My current settings are CRF=1 because 0 causes x264 to think its in lossless mode and then disables it, so I just use CRF=1 and the QP values were equal to CRF=0 anyway.

You should not rely on the avg QP-values reported by x264, especially when encoding like you do. When you use such low crf-values with a strict VBV-limit you can end up pretty much with CBR, this will make bit allocation across frames/segment worse, so complex frames wont benefit from high bit allocation that VBR allows for. So even though you get a lower average QP, you can have frames/section were the QP is much higher. This is also what I noticed on your previous SOL encode that I looked at, were complex burst of frames were having a lot of issues that wasnt present when I encoded it at 2pass vbr at a much lower avrage bitrate.

In general, its best to tweak settings in 2pass VBR at the same bitrate for all the runs, then, if you want to use crf, find a value were it ends up at the same average as your test were at. But in the case of bluray, in the real world you will always need to use a target bitrate as there is set size constrain for your video track (for the title to fit on disk with all the other stuff that needs to go in there), and in this case were we are not talking about actually putting titles on disk, and were saving space/bandwith isnt a criteria, just use 2pass VBR at a target that is in the high end of what is actually used by the format (so somewhere in the high 20s as I said previously).

I will test a buffsize of 30'000, hopefully, it won't pull the quality down too much. I've never tried using an open GOP on x265 or x264. It still requires a keyint of 24 though right? If so I don't see how an open GOP would change things though, what does it offer if the keyint must be 24?

Keyint sets the maxiumum distance of I frames, with open GOP those I frames can be non-IDR so you can reference frames across GOP-boundries. When using a short GOP (which 1s is) this can increase compression, I think i've seen papers that shows about 5-10% incease if i remember correctly, although it will greatly depend on the encoding scenario. TBH I have no idea how common open-gop is for actual commercial bluray authoring, but its in-spec to use.

[HD MOVIE SOURCE]

Originally Posted by excellentswordfight

I think you are meaning overflow? But the maxrate is the max-rate, it doesnt mean that you will constantly feed the buffer at that speed (the maxrate for bluray is based on the read-speed of the phyiscal discs, hence the lower value than the h264 spec allows), when both are set the encoder will make sure it doesnt over/underflow on decoders fit for those values. I never really understood why the bluray-spec dictates a lower bufsize, my best guest would that at the time it was set cheap decoders had a rather small buffer.



You should not rely on the avg QP-values reported by x264, especially when encoding like you do. When you use such low crf-values with a strict VBV-limit you can end up pretty much with CBR, this will make bit allocation across frames/segment worse, so complex frames wont benefit from high bit allocation that VBR allows for. So even though you get a lower average QP, you can have frames/section were the QP is much higher. This is also what I noticed on your previous SOL encode that I looked at, were complex burst of frames were having a lot of issues that wasnt present when I encoded it at 2pass vbr at a much lower avrage bitrate.

In general, its best to tweak settings in 2pass VBR at the same bitrate for all the runs, then, if you want to use crf, find a value were it ends up at the same average as your test were at. But in the case of bluray, in the real world you will always need to use a target bitrate as there is set size constrain for your video track (for the title to fit on disk with all the other stuff that needs to go in there), and in this case were we are not talking about actually putting titles on disk, and were saving space/bandwith isnt a criteria, just use 2pass VBR at a target that is in the high end of what is actually used by the format (so somewhere in the high 20s as I said previously).


Keyint sets the maxiumum distance of I frames, with open GOP those I frames can be non-IDR so you can reference frames across GOP-boundries. When using a short GOP (which 1s is) this can increase compression, I think i've seen papers that shows about 5-10% incease if i remember correctly, although it will greatly depend on the encoding scenario. TBH I have no idea how common open-gop is for actual commercial bluray authoring, but its in-spec to use.

My assumption is that 4K Blu-ray also has an open-GOP capability. Was there a white paper released for 4K Blu-ray? I'd like to see what the limitations are, and what needs to be included into every encode, and bframe limitations.

I wouldn't encode in the high 20 Mbps. I'm only interested in the absolutely maximum bit-rate the format can achieve. Spears and Munsil demo scenes on Blu-ray look to be around 38 Mbps average bit-rate and I've seen max peaks at 69 Mbps, and that's on a released Authored Blu-ray disc. I've seen some Kino Blu-ray discs that were constantly hitting 45 Mbps, then go down then burst back up to 45 and 52 Mbps. CRF=1 doesn't make the encode like constant bit-rate. It's constantly going from as low as 22 Mbps to 52 Mbps, but mostly around 36 Mbps, it's nothing like constant bit-rate. There's enough bandwidth even with Blu-ray to allow high data rate scenes and low. A low CRF doesn't seem to over-allocate bits I've found. I use this method on x265 also and the encodes are incredible.

I don't believe in holding back bit-rate for any reason on a Physical Format. Even on 4K Blu-ray I want people to target 98 Mbps unless they're actually capped for disc space, only then should the bit-rates come down. I don't believe in encoding to a target quality, only the highest possible is all I care about. High 20 Mbps is not good enough. Just my opinion.

I've encoded using CRF and 2-pass encoding, and CRF with a bit-rate cap, in my opinion, is better. I think you get improvements in low movement and dark areas too, where 2-pass encodes would lower the bit-rates too much in low movement and dark areas and reveal artifacts and break up. I've seen it, Blu-ray after Blu-ray, and it's not good enough. As long as the CRF maxrate cap allows it to keep in-spec, that's all I care about. Even in 2023, most Blu-ray encodes are just not good enough. Thankfully 4K Blu-ray encodes are improving significantly over the last year.


I do need your help, and one thing I've noticed is that my encodes using x264 have an interlaced or anti-aliasing look to them. Does --fake-interlaced or --pic-struct have to be included to avoid that? I encoded the same content using x264 and x265, and x265 looked as it should, perfect, but x264 looks wrong, almost like it's upscaled. I can't see this on my small monitor, but when I play it back on my 77-inch LGC9 and Panasonic UB9000, I can see it. I thought I must have downloaded an interlaced file by mistake, but I can't seem to get away from this look. I need to get this fixed asap.

So I check on my Panasonic UB9000 and it's outputting it as 60 frames per second. I checked on my Reavon x200 and that outputs 24 frames per second. I'm not sure whats causing this. Movies playback at 24 fps fine, but on x264 they're playing back at 60p. x265 MKV files play fine at 24p also. I might try mp4 and see if it's some type of restriction of the player not being able to play x264 in an MKV container.

Media Info Reports
Frame rate mode : Constant
Frame rate : 23.976 FPS

I will see if it's the container type first, and some type of player restriction first. This is annoying me.

I hope --fake-interlaced or --pic-struct will be what does it or is there something else that could also help?

I tried encoding without my settings, and just placebo and it still had that look, so I'm really trying to get this fixed.


EDIT - Okay, so I figured out it's the Panasonic UB9000 convert 1080p/24 into 1080/60 for some reason. It doesn't do it with 4K HEVC content. I've tested this on 2 other players and 1080p/24 plays properly.

The manual for the UB9000 says
Some MKV/mp4/MPEG files
may not be played back
correctly, depending on
the video resolution,
frame rate condition and
so on.

I'm just gonna have to consider this a compatibility issue with this player.

[excellentswordfight]

Originally Posted by HD MOVIE SOURCE

My assumption is that 4K Blu-ray also has an open-GOP capability. Was there a white paper released for 4K Blu-ray? I'd like to see what the limitations are, and what needs to be included into every encode, and bframe limitations.

Here you go:

https://web.archive.org/web/20160605...per_150724.pdf

I wouldn't encode in the high 20 Mbps. I'm only interested in the absolutely maximum bit-rate the format can achieve. Spears and Munsil demo scenes on Blu-ray look to be around 38 Mbps average bit-rate and I've seen max peaks at 69 Mbps, and that's on a released Authored Blu-ray disc. I've seen some Kino Blu-ray discs that were constantly hitting 45 Mbps, then go down then burst back up to 45 and 52 Mbps. CRF=1 doesn't make the encode like constant bit-rate. It's constantly going from as low as 22 Mbps to 52 Mbps, but mostly around 36 Mbps, it's nothing like constant bit-rate. There's enough bandwidth even with Blu-ray to allow high data rate scenes and low. A low CRF doesn't seem to over-allocate bits I've found. I use this method on x265 also and the encodes are incredible.

I don't believe in holding back bit-rate for any reason on a Physical Format. Even on 4K Blu-ray I want people to target 98 Mbps unless they're actually capped for disc space, only then should the bit-rates come down. I don't believe in encoding to a target quality, only the highest possible is all I care about. High 20 Mbps is not good enough. Just my opinion.

I've encoded using CRF and 2-pass encoding, and CRF with a bit-rate cap, in my opinion, is better. I think you get improvements in low movement and dark areas too, where 2-pass encodes would lower the bit-rates too much in low movement and dark areas and reveal artifacts and break up. I've seen it, Blu-ray after Blu-ray, and it's not good enough. As long as the CRF maxrate cap allows it to keep in-spec, that's all I care about. Even in 2023, most Blu-ray encodes are just not good enough. Thankfully 4K Blu-ray encodes are improving significantly over the last year.

I'm fully aware of the bitrate levels of blurays, as I have been re-encoding blurays for over a decade, as well as encoded video for physical discs. But you are either not listening or not understanding what im trying to tell you, maybe an image might explain it better:



The lower image is using your method of crf=1, the image above is a 2pass VBR at a lower average bitrate. As you can see the 2pass VBR encode is allowing higher spikes than the crf encode. This results in fewer frames with high QP values (above 20 in this case) even though the average is higher, and this is why Im saying that you should be cautious of using average qp as a metric when you are vbv-limited most of the time, cause it wont be seen there (one could argue of using qp as a metric at all in this case, but that is another discussion). You are absolutely free of using your encode method if you get good results for you, but you should be aware that there are drawbacks of doing it. Cause you can (note can, and not will, and this is was my phrasing in the previous post as well) end up in scenarios with worse quality for complex frames/sequences.

And when it comes to the bitrate, Im not saying that using something in the high 20s or whatever will give you better results just cause of the scenario above (the scenario above is independent this discussion), I think this is more that I still dont grasp what you are trying to achieve. If quality is the priority, I dont see why one should shot himself in the foot by being limited by the bluray-spec (cause it will pretty much only hurt quality for no reason if you dont want to create bluray-discs), if its a case of playback compliance, a big majority of playback devices will handle file based playback just fine if you stay within the relevant profile and level spec of avc/hevc. And if the interest is tuning an encoder to produce the best possible image for movies on blurays, you will rarely be able to use this method cause in the real world you have size target as is needs to fit it on disk, hence why it makes more sense to tune it for a more typical scenario (bitrate). But sorry if im out of line here, you do you, I just dont get it I guess.

I do need your help, and one thing I've noticed is that my encodes using x264 have an interlaced or anti-aliasing look to them. Does --fake-interlaced or --pic-struct have to be included to avoid that? I encoded the same content using x264 and x265, and x265 looked as it should, perfect, but x264 looks wrong, almost like it's upscaled. I can't see this on my small monitor, but when I play it back on my 77-inch LGC9 and Panasonic UB9000, I can see it. I thought I must have downloaded an interlaced file by mistake, but I can't seem to get away from this look. I need to get this fixed asap.

--fake-interlaced is used for 25/30p content as the bluray format doesnt support it, so its used pretty much as a workaround were you flag progressive content as interlaced as it supports 50/60i. If you are encoding 24p content it isnt relevant.

But not sure whats going on for you there, it seems odd that that device wouldnt be able to play back 1080p24 AVC files correctly.