New To x264 - Coming From x265 - High Detail - Bframe Quality and more...

[HD MOVIE SOURCE]

Hi,

I'm new to x264, and I'm coming from x265. I'm just wondering if any of the settings from x265 have any equivalence to x264?

The first thing I noticed was that x264 looks over-sharpened and edgy. So, I found that no-psy gives a similar look to x265, but even then there is some edge. With that said are there any settings in x264 that control detail outside of psy controls? I've read that the deadzones can control fine detail. I'm after as much detail as the encoder will allow.
Is deadzone-inter=0 and deadzone-intra=0 absolute maximum detail?
Are there any other settings that specifically go after detail?

Like x265, I keep to uhd-bd=1 just to give me a sense of a spec to go after. With x264 I'm using bluray-compat=1. I do this to learn how to maximize encodes for a particular constraint. I enjoy doing this. Are there any restrictions using bluray-compat=1?
I know that Blu-ray can only use 3 bframes, which is honestly fine. Also it can only use as high as b-pyramid=1, which I've found to be a quality loss anyway, so I just use b-pyramid=0.

Are there any other specific settings that are required for Blu-ray compatibility? How many refs is it 3 or 4?

With x265 use can use ctu=64, would an equivalent setting in x264 be 8x8dct=1? Is that as high in blocks as we can go?

I read here: http://www.chaneru.com/Roku/HLS/X264_Settings.htm that slices on Blu-ray should be set to 4, is this accurate?

Are lookahead_threads=0 the same as x265? I typically set those to zero.


Another thing I'd like to know, exactly which settings increase bframe and pframe quality? Using x265 we have b-adapt=2, b-intra, b-pyramid, weightb, weightp.
I've seen that x264 has some similar options, but it does haven't b-intra, are there any more settings where I can get even more quality out of bframes?


Thanks for reading, any info is welcome. I'm basically trying to get as much detail without that artificial overshrpened look, and keeping to a restriction like the Blu-ray format because it helps me try to maximize what I have. I prefer to learn that way rather than just maxing everything out.

[microchip8]

x264 produces better fine details, even without tweaks - it's the other way around. People coming from x264 often complain that x265 looks too soft. You have to tune the hell out of x265 to come close to the x264 detail preservation. That said, If you tweak x264 too much, it'll look over-sharpen. I suggest go with the defaults of a specific high-quality preset like 'slow' or 'veryslow'. The only thing that's overkill in these presets is the ref count. For live-action content, anything more than 6 can degrade quality. For anime, up-to 16 can be useful. The only thing I'd do is lower deblock a bit (-1,-1 is fine) and disable decimate with --no-dct-decimate. x264 has in addition to presets, also psy tunings. Might try them.

Regarding Blu-ray, the only other thing you missed is lower amount of keyint and a closed GOP.

[HD MOVIE SOURCE]

Originally Posted by microchip8

x264 produces better fine details, even without tweaks - it's the other way around. People coming from x264 often complain that x265 looks too soft. You have to tune the hell out of x265 to come close to the x264 detail preservation. That said, If you tweak x264 too much, it'll look over-sharpen. I suggest go with the defaults of a specific high-quality preset like 'slow' or 'veryslow'. The only thing that's overkill in these presets is the ref count. For live-action content, anything more than 6 can degrade quality. For anime, up-to 16 can be useful. The only thing I'd do is lower deblock a bit (-1,-1 is fine) and disable decimate with --no-dct-decimate. x264 has in addition to presets, also psy tunings. Might try them.

Regarding Blu-ray, the only other thing you missed is lower amount of keyint and a closed GOP.

This is true, I spent a long time trying to get x265 from being so soft to having as much detail as possible. It takes a lot to get there. I noticed that x264 had too much detail though, and maybe in an artificial way. It almost looked like edge enhancement, but maybe there's a fine balance there with the psy adjustments. I think it will only need small number for psy. I will have to do some more testing.

Right now I'm using no-deblock with CRF=0, with a bandwidth restriction of vbv-maxrate=40000 and vbv-bufsize=40000. I don't see any artifacts of blocking of any kind. Does x264 suffer from image softening when using deblock? This is one reason I didn't want to use it. It appears at high bit-rates that its not needed as much.

[excellentswordfight]

Originally Posted by HD MOVIE SOURCE

Are there any other specific settings that are required for Blu-ray compatibility? How many refs is it 3 or 4?

Its 4 for 1080p (it will be limited for you when setting bluray-compat),, you can find find the detailed speccs here (see annex a):

https://web.archive.org/web/20151028...per_110712.pdf

There is also a good writeup of the important parameters here:

https://forum.doom9.org/showthread.php?t=154533

In general, this is what you want to set for 1080p24:

--profile high --level 4.1 --bluray-compat --vbv-maxrate 40000 --vbv-bufsize 30000 --keyint 24 --slices 4

Open-GOP is also allowed according to the specs, so you can also use --open-gop, its usually rather beneficial at small GOP-sizes (which the specc mandates). Also note that the bufsize limit of bluray is 30Mbps, not 40.

Use that with --preset veryslow & --tune film and nothing else with something like 2pass vbr @ the mid to high 20s, as starting point, (and if you dont wanna use it as a starting point, atleast use it as reference point to compare your encoding to). Analyze the produced image and start to evaluate what you are not happy with/what issues you are having and start tuning based on that. This should not produce a oversharpened look, or soft look, using deblocking strength from -1 to -3 is rather common with x264 and should be a rather decent balance for most live-action content (the tune film preset uses -1).

And please always post your complete command line, otherwise its close to impossible to give feedback. But from the sound if its already rather unorthadox so it might be hard to give advice as the encoder will operate in a very different way to what most people are used to ("no-deblock with CRF=0, with a bandwidth restriction of vbv-maxrate=40000 and vbv-bufsize=40000").

[HD MOVIE SOURCE]

Originally Posted by excellentswordfight

Its 4 for 1080p (it will be limited for you when setting bluray-compat),, you can find find the detailed speccs here (see annex a):

https://web.archive.org/web/20151028...per_110712.pdf

There is also a good writeup of the important parameters here:

https://forum.doom9.org/showthread.php?t=154533

In general, this is what you want to set for 1080p24:

--profile high --level 4.1 --bluray-compat --vbv-maxrate 40000 --vbv-bufsize 30000 --keyint 24 --slices 4

Open-GOP is also allowed according to the specs, so you can also use --open-gop, its usually rather beneficial at small GOP-sizes (which the specc mandates). Also note that the bufsize limit of bluray is 30Mbps, not 40.

Use that with --preset veryslow & --tune film and nothing else with something like 2pass vbr @ the mid to high 20s, as starting point, (and if you dont wanna use it as a starting point, atleast use it as reference point to compare your encoding to). Analyze the produced image and start to evaluate what you are not happy with/what issues you are having and start tuning based on that. This should not produce a oversharpened look, or soft look, using deblocking strength from -1 to -3 is rather common with x264 and should be a rather decent balance for most live-action content (the tune film preset uses -1).

And please always post your complete command line, otherwise its close to impossible to give feedback. But from the sound if its already rather unorthadox so it might be hard to give advice as the encoder will operate in a very different way to what most people are used to ("no-deblock with CRF=0, with a bandwidth restriction of vbv-maxrate=40000 and vbv-bufsize=40000").

Thanks for the feedback and the links, Ive been looking for things like this but came up with nothing. I really appreciate it.

If you use a buffsize of 30 Mbps and a bit-rate of 40 Mbps, would that create underflow?

My current settings are CRF=1 because 0 causes x264 to think its in lossless mode and then disables it, so I just use CRF=1 and the QP values were equal to CRF=0 anyway.

My current settings are being tested right now, but I' also done. I use very slow because it gives lower QP values than placebo for some reason if using the same settings.

This is the highest quality Ive been able to get so far using Vidcoder.

bluray-compat=1:no-open-gop:min-keyint=1:keyint=24:rc-lookahead=24:slices=4:scenecut=40:vbv-maxrate=40000:vbv-bufsize=40000: pbratio=1.00:ipratio=1.00:ref=3:constrained-intra=0:mixed-refs=1:bframes=3:b-adapt=2:b-pyramid=0:weightp=1:weightb=1:no-deblock:subme=11: psy=0:mbtree=0:deadzone-inter=0:deadzone-intra=0:no-dct-decimate:cabac=1:8x8dct=1:direct=auto:analyse=all:me=tesa:merange=24:chroma-me=0:trellis=2:fast-pskip=0: partitions=all:aq-mode=3:aq-strength=1.7

An example of the frame quality on a prores trailer I'm getting so far and I haven't finished optimizing quite yet.
HB x264 [info]: frame I:181 Avg QP: 7.99 size:303143
HB x264 [info]: frame P:1259 Avg QP: 8.94 size:211721
HB x264 [info]: frame B:2224 Avg QP: 8.76 size:159667

It's interesting how chroma-me being turned off increased quality by a little. And after testing all the AQ modes, aq-mode 3 yielded the lowest QP, and I'm currently trying to find the best setting there. On x265, on grainless content, it was 1.7, so I'm currently testing that. b-pyramid I actually lose quality, which I found odd because it's a quality increase on x265. I've run tests with 1, 2 and 3 bframes, and all can look very good. But using b-adapt allows better control I think. I tried no-scenecut and I lost quality by quite a bit. The reason I tried that is that trailers have lots of scenecuts, but without a good reference frame, x264 seems to lose quality quite quickly. It needs that iframe, otherwise, the p and bframes lose quality. x265 seems to be able to get away with many more bframes without losing quality which is really nice, but requires so much CPU power.


I will test a buffsize of 30'000, hopefully, it won't pull the quality down too much. I've never tried using an open GOP on x265 or x264. It still requires a keyint of 24 though right? If so I don't see how an open GOP would change things though, what does it offer if the keyint must be 24?

Thanks.

[excellentswordfight]

Originally Posted by HD MOVIE SOURCE

Thanks for the feedback and the links, Ive been looking for things like this but came up with nothing. I really appreciate it.

If you use a buffsize of 30 Mbps and a bit-rate of 40 Mbps, would that create underflow?

I think you are meaning overflow? But the maxrate is the max-rate, it doesnt mean that you will constantly feed the buffer at that speed (the maxrate for bluray is based on the read-speed of the phyiscal discs, hence the lower value than the h264 spec allows), when both are set the encoder will make sure it doesnt over/underflow on decoders fit for those values. I never really understood why the bluray-spec dictates a lower bufsize, my best guest would that at the time it was set cheap decoders had a rather small buffer.

My current settings are CRF=1 because 0 causes x264 to think its in lossless mode and then disables it, so I just use CRF=1 and the QP values were equal to CRF=0 anyway.

You should not rely on the avg QP-values reported by x264, especially when encoding like you do. When you use such low crf-values with a strict VBV-limit you can end up pretty much with CBR, this will make bit allocation across frames/segment worse, so complex frames wont benefit from high bit allocation that VBR allows for. So even though you get a lower average QP, you can have frames/section were the QP is much higher. This is also what I noticed on your previous SOL encode that I looked at, were complex burst of frames were having a lot of issues that wasnt present when I encoded it at 2pass vbr at a much lower avrage bitrate.

In general, its best to tweak settings in 2pass VBR at the same bitrate for all the runs, then, if you want to use crf, find a value were it ends up at the same average as your test were at. But in the case of bluray, in the real world you will always need to use a target bitrate as there is set size constrain for your video track (for the title to fit on disk with all the other stuff that needs to go in there), and in this case were we are not talking about actually putting titles on disk, and were saving space/bandwith isnt a criteria, just use 2pass VBR at a target that is in the high end of what is actually used by the format (so somewhere in the high 20s as I said previously).

I will test a buffsize of 30'000, hopefully, it won't pull the quality down too much. I've never tried using an open GOP on x265 or x264. It still requires a keyint of 24 though right? If so I don't see how an open GOP would change things though, what does it offer if the keyint must be 24?

Keyint sets the maxiumum distance of I frames, with open GOP those I frames can be non-IDR so you can reference frames across GOP-boundries. When using a short GOP (which 1s is) this can increase compression, I think i've seen papers that shows about 5-10% incease if i remember correctly, although it will greatly depend on the encoding scenario. TBH I have no idea how common open-gop is for actual commercial bluray authoring, but its in-spec to use.

[HD MOVIE SOURCE]

Originally Posted by excellentswordfight

I think you are meaning overflow? But the maxrate is the max-rate, it doesnt mean that you will constantly feed the buffer at that speed (the maxrate for bluray is based on the read-speed of the phyiscal discs, hence the lower value than the h264 spec allows), when both are set the encoder will make sure it doesnt over/underflow on decoders fit for those values. I never really understood why the bluray-spec dictates a lower bufsize, my best guest would that at the time it was set cheap decoders had a rather small buffer.



You should not rely on the avg QP-values reported by x264, especially when encoding like you do. When you use such low crf-values with a strict VBV-limit you can end up pretty much with CBR, this will make bit allocation across frames/segment worse, so complex frames wont benefit from high bit allocation that VBR allows for. So even though you get a lower average QP, you can have frames/section were the QP is much higher. This is also what I noticed on your previous SOL encode that I looked at, were complex burst of frames were having a lot of issues that wasnt present when I encoded it at 2pass vbr at a much lower avrage bitrate.

In general, its best to tweak settings in 2pass VBR at the same bitrate for all the runs, then, if you want to use crf, find a value were it ends up at the same average as your test were at. But in the case of bluray, in the real world you will always need to use a target bitrate as there is set size constrain for your video track (for the title to fit on disk with all the other stuff that needs to go in there), and in this case were we are not talking about actually putting titles on disk, and were saving space/bandwith isnt a criteria, just use 2pass VBR at a target that is in the high end of what is actually used by the format (so somewhere in the high 20s as I said previously).


Keyint sets the maxiumum distance of I frames, with open GOP those I frames can be non-IDR so you can reference frames across GOP-boundries. When using a short GOP (which 1s is) this can increase compression, I think i've seen papers that shows about 5-10% incease if i remember correctly, although it will greatly depend on the encoding scenario. TBH I have no idea how common open-gop is for actual commercial bluray authoring, but its in-spec to use.

My assumption is that 4K Blu-ray also has an open-GOP capability. Was there a white paper released for 4K Blu-ray? I'd like to see what the limitations are, and what needs to be included into every encode, and bframe limitations.

I wouldn't encode in the high 20 Mbps. I'm only interested in the absolutely maximum bit-rate the format can achieve. Spears and Munsil demo scenes on Blu-ray look to be around 38 Mbps average bit-rate and I've seen max peaks at 69 Mbps, and that's on a released Authored Blu-ray disc. I've seen some Kino Blu-ray discs that were constantly hitting 45 Mbps, then go down then burst back up to 45 and 52 Mbps. CRF=1 doesn't make the encode like constant bit-rate. It's constantly going from as low as 22 Mbps to 52 Mbps, but mostly around 36 Mbps, it's nothing like constant bit-rate. There's enough bandwidth even with Blu-ray to allow high data rate scenes and low. A low CRF doesn't seem to over-allocate bits I've found. I use this method on x265 also and the encodes are incredible.

I don't believe in holding back bit-rate for any reason on a Physical Format. Even on 4K Blu-ray I want people to target 98 Mbps unless they're actually capped for disc space, only then should the bit-rates come down. I don't believe in encoding to a target quality, only the highest possible is all I care about. High 20 Mbps is not good enough. Just my opinion.

I've encoded using CRF and 2-pass encoding, and CRF with a bit-rate cap, in my opinion, is better. I think you get improvements in low movement and dark areas too, where 2-pass encodes would lower the bit-rates too much in low movement and dark areas and reveal artifacts and break up. I've seen it, Blu-ray after Blu-ray, and it's not good enough. As long as the CRF maxrate cap allows it to keep in-spec, that's all I care about. Even in 2023, most Blu-ray encodes are just not good enough. Thankfully 4K Blu-ray encodes are improving significantly over the last year.


I do need your help, and one thing I've noticed is that my encodes using x264 have an interlaced or anti-aliasing look to them. Does --fake-interlaced or --pic-struct have to be included to avoid that? I encoded the same content using x264 and x265, and x265 looked as it should, perfect, but x264 looks wrong, almost like it's upscaled. I can't see this on my small monitor, but when I play it back on my 77-inch LGC9 and Panasonic UB9000, I can see it. I thought I must have downloaded an interlaced file by mistake, but I can't seem to get away from this look. I need to get this fixed asap.

So I check on my Panasonic UB9000 and it's outputting it as 60 frames per second. I checked on my Reavon x200 and that outputs 24 frames per second. I'm not sure whats causing this. Movies playback at 24 fps fine, but on x264 they're playing back at 60p. x265 MKV files play fine at 24p also. I might try mp4 and see if it's some type of restriction of the player not being able to play x264 in an MKV container.

Media Info Reports
Frame rate mode : Constant
Frame rate : 23.976 FPS

I will see if it's the container type first, and some type of player restriction first. This is annoying me.

I hope --fake-interlaced or --pic-struct will be what does it or is there something else that could also help?

I tried encoding without my settings, and just placebo and it still had that look, so I'm really trying to get this fixed.


EDIT - Okay, so I figured out it's the Panasonic UB9000 convert 1080p/24 into 1080/60 for some reason. It doesn't do it with 4K HEVC content. I've tested this on 2 other players and 1080p/24 plays properly.

The manual for the UB9000 says
Some MKV/mp4/MPEG files
may not be played back
correctly, depending on
the video resolution,
frame rate condition and
so on.

I'm just gonna have to consider this a compatibility issue with this player.

[excellentswordfight]

Originally Posted by HD MOVIE SOURCE

My assumption is that 4K Blu-ray also has an open-GOP capability. Was there a white paper released for 4K Blu-ray? I'd like to see what the limitations are, and what needs to be included into every encode, and bframe limitations.

Here you go:

https://web.archive.org/web/20160605...per_150724.pdf

I wouldn't encode in the high 20 Mbps. I'm only interested in the absolutely maximum bit-rate the format can achieve. Spears and Munsil demo scenes on Blu-ray look to be around 38 Mbps average bit-rate and I've seen max peaks at 69 Mbps, and that's on a released Authored Blu-ray disc. I've seen some Kino Blu-ray discs that were constantly hitting 45 Mbps, then go down then burst back up to 45 and 52 Mbps. CRF=1 doesn't make the encode like constant bit-rate. It's constantly going from as low as 22 Mbps to 52 Mbps, but mostly around 36 Mbps, it's nothing like constant bit-rate. There's enough bandwidth even with Blu-ray to allow high data rate scenes and low. A low CRF doesn't seem to over-allocate bits I've found. I use this method on x265 also and the encodes are incredible.

I don't believe in holding back bit-rate for any reason on a Physical Format. Even on 4K Blu-ray I want people to target 98 Mbps unless they're actually capped for disc space, only then should the bit-rates come down. I don't believe in encoding to a target quality, only the highest possible is all I care about. High 20 Mbps is not good enough. Just my opinion.

I've encoded using CRF and 2-pass encoding, and CRF with a bit-rate cap, in my opinion, is better. I think you get improvements in low movement and dark areas too, where 2-pass encodes would lower the bit-rates too much in low movement and dark areas and reveal artifacts and break up. I've seen it, Blu-ray after Blu-ray, and it's not good enough. As long as the CRF maxrate cap allows it to keep in-spec, that's all I care about. Even in 2023, most Blu-ray encodes are just not good enough. Thankfully 4K Blu-ray encodes are improving significantly over the last year.

I'm fully aware of the bitrate levels of blurays, as I have been re-encoding blurays for over a decade, as well as encoded video for physical discs. But you are either not listening or not understanding what im trying to tell you, maybe an image might explain it better:



The lower image is using your method of crf=1, the image above is a 2pass VBR at a lower average bitrate. As you can see the 2pass VBR encode is allowing higher spikes than the crf encode. This results in fewer frames with high QP values (above 20 in this case) even though the average is higher, and this is why Im saying that you should be cautious of using average qp as a metric when you are vbv-limited most of the time, cause it wont be seen there (one could argue of using qp as a metric at all in this case, but that is another discussion). You are absolutely free of using your encode method if you get good results for you, but you should be aware that there are drawbacks of doing it. Cause you can (note can, and not will, and this is was my phrasing in the previous post as well) end up in scenarios with worse quality for complex frames/sequences.

And when it comes to the bitrate, Im not saying that using something in the high 20s or whatever will give you better results just cause of the scenario above (the scenario above is independent this discussion), I think this is more that I still dont grasp what you are trying to achieve. If quality is the priority, I dont see why one should shot himself in the foot by being limited by the bluray-spec (cause it will pretty much only hurt quality for no reason if you dont want to create bluray-discs), if its a case of playback compliance, a big majority of playback devices will handle file based playback just fine if you stay within the relevant profile and level spec of avc/hevc. And if the interest is tuning an encoder to produce the best possible image for movies on blurays, you will rarely be able to use this method cause in the real world you have size target as is needs to fit it on disk, hence why it makes more sense to tune it for a more typical scenario (bitrate). But sorry if im out of line here, you do you, I just dont get it I guess.

I do need your help, and one thing I've noticed is that my encodes using x264 have an interlaced or anti-aliasing look to them. Does --fake-interlaced or --pic-struct have to be included to avoid that? I encoded the same content using x264 and x265, and x265 looked as it should, perfect, but x264 looks wrong, almost like it's upscaled. I can't see this on my small monitor, but when I play it back on my 77-inch LGC9 and Panasonic UB9000, I can see it. I thought I must have downloaded an interlaced file by mistake, but I can't seem to get away from this look. I need to get this fixed asap.

--fake-interlaced is used for 25/30p content as the bluray format doesnt support it, so its used pretty much as a workaround were you flag progressive content as interlaced as it supports 50/60i. If you are encoding 24p content it isnt relevant.

But not sure whats going on for you there, it seems odd that that device wouldnt be able to play back 1080p24 AVC files correctly.

[HD MOVIE SOURCE]

Originally Posted by excellentswordfight

Here you go:

https://web.archive.org/web/20160605...per_150724.pdf



I'm fully aware of the bitrate levels of blurays, as I have been re-encoding blurays for over a decade, as well as encoded video for physical discs. But you are either not listening or not understanding what im trying to tell you, maybe an image might explain it better:



The lower image is using your method of crf=1, the image above is a 2pass VBR at a lower average bitrate. As you can see the 2pass VBR encode is allowing higher spikes than the crf encode. This results in fewer frames with high QP values (above 20 in this case) even though the average is higher, and this is why Im saying that you should be cautious of using average qp as a metric when you are vbv-limited most of the time, cause it wont be seen there (one could argue of using qp as a metric at all in this case, but that is another discussion). You are absolutely free of using your encode method if you get good results for you, but you should be aware that there are drawbacks of doing it. Cause you can (note can, and not will, and this is was my phrasing in the previous post as well) end up in scenarios with worse quality for complex frames/sequences.

And when it comes to the bitrate, Im not saying that using something in the high 20s or whatever will give you better results just cause of the scenario above (the scenario above is independent this discussion), I think this is more that I still dont grasp what you are trying to achieve. If quality is the priority, I dont see why one should shot himself in the foot by being limited by the bluray-spec (cause it will pretty much only hurt quality for no reason if you dont want to create bluray-discs), if its a case of playback compliance, a big majority of playback devices will handle file based playback just fine if you stay within the relevant profile and level spec of avc/hevc. And if the interest is tuning an encoder to produce the best possible image for movies on blurays, you will rarely be able to use this method cause in the real world you have size target as is needs to fit it on disk, hence why it makes more sense to tune it for a more typical scenario (bitrate). But sorry if im out of line here, you do you, I just dont get it I guess.


--fake-interlaced is used for 25/30p content as the bluray format doesnt support it, so its used pretty much as a workaround were you flag progressive content as interlaced as it supports 50/60i. If you are encoding 24p content it isnt relevant.

But not sure whats going on for you there, it seems odd that that device wouldnt be able to play back 1080p24 AVC files correctly.

Thank you, much appreciated for the link. I'll have a read of that soon.

I'm personally a fan of doing it this way, using CRF. I do like the quality that I'm getting on both x265 and x264.
If you'd like to see my x265 encode for Sol Levante, I'd like to know what you think of it: https://archive.org/details/sol-leva...lix-short-x265
It has a look that I want from x265, especially considering that it's a soft encoder. My goal was to achieve as much detail as x265 allows. I think I achieved that.

Yeah, I've noticed that some Blu-ray and 4K Players do not like x264 1080p/24, and convert it to 1080p/60. Windows players don't do it, some 4K Players like my Reavon x200 don't do it, and maybe some smart TVs wouldn't do it either. TVs like my LGC9 seem to play files very well, but I'll have to check that out.

I've figured out the rd levels using x264 on film-based material to be 1.57. x264 seems to have some ringing anyway, and using no-psy makes the image too soft even for x264. I also prefer to use equal IP and PB ratios for more consistent frame quality. It really helps with film grain consistency, and this is another thing that I prefer.

My final settings after testing and tuning are...

CRF=1, Very Slow (Better than Placebo)

bluray-compat=1:no-open-gop:min-keyint=1:keyint=24:rc-lookahead=24:slices=4:scenecut=40:vbv-maxrate=40000:vbv-bufsize=30000: pbratio=1.00:ipratio=1.00:ref=4:constrained-intra=0:mixed-refs=1:bframes=3:b-adapt=2:b-pyramid=0:weightp=1:weightb=1:no-deblock: psy-rd=1.57,0.00:subme=11:mbtree=0:deadzone-inter=0:deadzone-intra=0:no-dct-decimate:cabac=1:8x8dct=1:direct=auto:analyse=all:me=tesa:merange=24:chroma-me=0:trellis=2:fast-pskip=0: partitions=all:aq-mode=3:aq-strength=1.6

I've been testing it on prores trailers with film grain and I think it looks excellent.

This is the frame quality from Oppenheimer Trailer 1. Pretty happy with it.
HB x264 [info]: frame I:140 Avg QP: 9.78 size:328976
HB x264 [info]: frame P:1048 Avg QP:10.89 size:211909
HB x264 [info]: frame B:1636 Avg QP:10.43 size:156733

[excellentswordfight]

Originally Posted by HD MOVIE SOURCE

It has a look that I want from x265, especially considering that it's a soft encoder. My goal was to achieve as much detail as x265 allows. I think I achieved that.

Well its anime at 70Mbps, so ofc, for the most part it looks fine, but it took me like 10s to find stuff like this, that looks horrible (a lot of the complex frames have this issue):

https://i.imgur.com/9vOUrhN.png

This is the same part with x265 with pretty much stock settings; preset slow at 50mbps with some minor tweaks (also bluray-compliant)

https://i.imgur.com/FKnfT21.png

The parts of the video without this issue of visible CU-blocks it looks virtually the same, there is no relevant difference in sharpness/detail between the versions.

My final settings after testing and tuning are...

CRF=1, Very Slow (Better than Placebo)

bluray-compat=1:no-open-gop:min-keyint=1:keyint=24:rc-lookahead=24:slices=4:scenecut=40:vbv-maxrate=40000:vbv-bufsize=30000: pbratio=1.00:ipratio=1.00:ref=4:constrained-intra=0:mixed-refs=1:bframes=3:b-adapt=2:b-pyramid=0:weightp=1:weightb=1:no-deblock: psy-rd=1.57,0.00:subme=11:mbtree=0:deadzone-inter=0:deadzone-intra=0:no-dct-decimate:cabac=1:8x8dct=1:direct=auto:analyse=all:me=tesa:merange=24:chroma-me=0:trellis=2:fast-pskip=0: partitions=all:aq-mode=3:aq-strength=1.6

I've been testing it on prores trailers with film grain and I think it looks excellent.

This is the frame quality from Oppenheimer Trailer 1. Pretty happy with it.
HB x264 [info]: frame I:140 Avg QP: 9.78 size:328976
HB x264 [info]: frame P:1048 Avg QP:10.89 size:211909
HB x264 [info]: frame B:1636 Avg QP:10.43 size:156733

I tried those out on Tears of Steel, seemed good enough, resulted in pretty much an transparent encode. I compared it to the VBR encode I did earlier for the VBR-graph, with pretty much just veryslow tune film @ 30Mbps and they look pretty much the same, found frames though were some heavy zooming and pixel peaking favored 30Mbps encode with the stock settings (issues with blocking). Although I quite like ToS as a source (the best freely available lossless source that is a good reference for your standard SDR movie), it dont have any really complex scenes, so for this case it simply doesnt pushes the encoder enough with these bitrate-levels (But I do get higher QP-values with your settings for this than you get from that trailer so thats at least an indicator that its harder to compress than that). But I have a feeling that no-deblock can cause some nasty side-effects when you encounter something really complex were the high bitrate cant handle it by brute force, other than that the settings doesnt look too odd, maybe a few here and there (why disable ME on the chroma plane? And isnt b-pyramid strict allowed?)

[HD MOVIE SOURCE]

Originally Posted by excellentswordfight

Well its anime at 70Mbps, so ofc, for the most part it looks fine, but it took me like 10s to find stuff like this, that looks horrible (a lot of the complex frames have this issue):

https://i.imgur.com/9vOUrhN.png

This is the same part with x265 with pretty much stock settings; preset slow at 50mbps with some minor tweaks (also bluray-compliant)

https://i.imgur.com/FKnfT21.png

The parts of the video without this issue of visible CU-blocks it looks virtually the same, there is no relevant difference in sharpness/detail between the versions.


I tried those out on Tears of Steel, seemed good enough, resulted in pretty much an transparent encode. I compared it to the VBR encode I did earlier for the VBR-graph, with pretty much just veryslow tune film @ 30Mbps and they look pretty much the same, found frames though were some heavy zooming and pixel peaking favored 30Mbps encode with the stock settings (issues with blocking). Although I quite like ToS as a source (the best freely available lossless source that is a good reference for your standard SDR movie), it dont have any really complex scenes, so for this case it simply doesnt pushes the encoder enough with these bitrate-levels (But I do get higher QP-values with your settings for this than you get from that trailer so thats at least an indicator that its harder to compress than that). But I have a feeling that no-deblock can cause some nasty side-effects when you encounter something really complex were the high bitrate cant handle it by brute force, other than that the settings doesnt look too odd, maybe a few here and there (why disable ME on the chroma plane? And isnt b-pyramid strict allowed?)

Thanks for the reply. I disabled ME on Chroma because I was getting higher QP values from it. When I run tests and get to the end of trying to find optimal settings, I only use settings that yield the lowest QP levels. ME Chroma, for some reason, caused QP levels to raise, and I didn't see any noticeable difference from actually using it. The QP difference was only half a QP on average, but, again, when I'm trying to get as low as I can, I test everything, and if it affects quality in any way I try to turn them off or on.

Same with b-pyramid. I used strict and it raised QP values, and I tried to think about why it did this. My first thought is that B frame quality using x264 is not as high as x265, and reference frames are better spent on p frames rather than Bframes. Turning b-pyramid off actually increased b frame quality for me. I assume 2 is much better than 1, I didn't try it because Blu-ray can't use it, so I feel it's better just turned off.

I've just started putting out some x264 encodes on the archive. I'd like to know what you think about this one. It's an Oppenheimer trailer and I think it looks really good. It looks as though the buff size of 30Mbps seems to affect burst bit-rate scenarios. At 40 Mbps it could burst up to 60 Mbps or so, but now using a buffsize of 30 Mbps it seems to burst up to around 44 Mbps ish which is about what I see the very best Blu-rays doing.
Let me know what you think, it's the MKV file: https://archive.org/details/oppenhei...l-trailer-x264

I think you're right, my method is about always using the highest bit-rate a platform has to offer, and sometimes even a small amount of deblock or even intra-smoothing may help. Even though a screenshot looks as though it reveals something that looks equal in detail, when I see it on a screen, even the smallest amount of blurring of softness can affect the way it looks. I view it on an OLED and I can see when things like sao are turned on. But I can't see that from a still image, so I tend to take screenshots with a grain of salt. On anime like this though, I've used screenshots to improve blocking and artifacts. However, it couldn't be seen when moving at all.
How I improved it was using simply using a psy-rdoq of 0.4, more bframes, and adaptive bframes. I'm using 8bframes. I've seen 4K discs use 7 so I just assumed that they'd be able to use 8. The frame that I was trying to clean up was an iframe from a scenecut, so there was a previous iframe that was close, and then immediately had a scene cut maybe after a couple of frames. The thought process here was that using more bframes lowers bit-rates and allows for better iframe quality because more bits could be used on the iframe because of the savings. That's my thought and it worked.

Do you use any specific software to inspect quality? Also, what tools are you using to see average bit-rate peaks in your screenshots?

[excellentswordfight]

I just wanted to point out, one last time, the danger of your QP-approach when vbv limited. Its very bad to make decisions based on the average; low values will look indistinguishable to each other, so you can get results were get noticeable worse results on some sequences but still lowering the average just cause a big portion of the video gets a slightly lower value, a portion that already achieved transparency. This will be especially true once you start to encode stuff that have segments that are actually hard to compress, and were you dont have any bandwith headroom, so if you dont plan doing very custom per-title-encoding, please at least use a more worst case source and if you wanna use QP as a metric for these kinds of decisions use the stats file so you can analyze the spread and look at the highs. Like on that SoL encode, I was actually able to spot the CU-boundry boxes issue without any zooming shenanigans, I even saw it on a "normal" frame that didnt have any crazy complexity to it, it was just a scenecut change... But in the end, if you are encoding for yourself, and if you are happy, good for you, but I woudlnt pass anything with this issue as a quality encode at this bandwidth, and it woudlnt (shoudlnt) pass QC.

Anyway, I think I already made most of the points I was trying to make, so Im gonna drop out here as it doesnt seem to be very productive as it seems like we are going circles. But I do agree that quality during normal playback is most important, and not stills, but that doesnt mean that stills are a bad tool for analysis, and SoL can be very much encoded at the same level of sharpness without those CU-boarders.

[HD MOVIE SOURCE]

Originally Posted by excellentswordfight

I just wanted to point out, one last time, the danger of your QP-approach when vbv limited. Its very bad to make decisions based on the average; low values will look indistinguishable to each other, so you can get results were get noticeable worse results on some sequences but still lowering the average just cause a big portion of the video gets a slightly lower value, a portion that already achieved transparency. This will be especially true once you start to encode stuff that have segments that are actually hard to compress, and were you dont have any bandwith headroom, so if you dont plan doing very custom per-title-encoding, please at least use a more worst case source and if you wanna use QP as a metric for these kinds of decisions use the stats file so you can analyze the spread and look at the highs. Like on that SoL encode, I was actually able to spot the CU-boundry boxes issue without any zooming shenanigans, I even saw it on a "normal" frame that didnt have any crazy complexity to it, it was just a scenecut change... But in the end, if you are encoding for yourself, and if you are happy, good for you, but I woudlnt pass anything with this issue as a quality encode at this bandwidth, and it woudlnt (shoudlnt) pass QC.

Anyway, I think I already made most of the points I was trying to make, so Im gonna drop out here as it doesnt seem to be very productive as it seems like we are going circles. But I do agree that quality during normal playback is most important, and not stills, but that doesnt mean that stills are a bad tool for analysis, and SoL can be very much encoded at the same level of sharpness without those CU-boarders.

How do you improve the CU-boundry issue and how do you spot it?

[rwill]

Originally Posted by HD MOVIE SOURCE

Let me know what you think, it's the MKV file: https://archive.org/details/oppenhei...l-trailer-x264

Are you using archive.org as some sort of personal file hosting service?

[benwaggoner]

Originally Posted by excellentswordfight

Well its anime at 70Mbps, so ofc, for the most part it looks fine, but it took me like 10s to find stuff like this, that looks horrible (a lot of the complex frames have this issue):

https://i.imgur.com/9vOUrhN.png

This is the same part with x265 with pretty much stock settings; preset slow at 50mbps with some minor tweaks (also bluray-compliant)

https://i.imgur.com/FKnfT21.png

The parts of the video without this issue of visible CU-blocks it looks virtually the same, there is no relevant difference in sharpness/detail between the versions.

Note that this is from Sol Levante, which isn't a real anime title, but a compression test clip, and arguably the most challenging commonly used source content in existence. It's nice to have 4K HDR anime, but I strongly caution against over-extrapolating from results from this very uniquely challenging clip.

Also, SDR PNG frame comparisons from HDR moving images aren't really apples to apples. Motion can mask all sorts of compression issues, and highlight others that frame comparisons can miss. Quality during 1x speed playback is the important optimization goal.

That said, that sequence from Sol Levante looks gnarly in pretty much every encoder I've tried.

[benwaggoner]

Originally Posted by excellentswordfight

I just wanted to point out, one last time, the danger of your QP-approach when vbv limited. Its very bad to make decisions based on the average; low values will look indistinguishable to each other, so you can get results were get noticeable worse results on some sequences but still lowering the average just cause a big portion of the video gets a slightly lower value, a portion that already achieved transparency. This will be especially true once you start to encode stuff that have segments that are actually hard to compress, and were you dont have any bandwith headroom, so if you dont plan doing very custom per-title-encoding, please at least use a more worst case source and if you wanna use QP as a metric for these kinds of decisions use the stats file so you can analyze the spread and look at the highs. Like on that SoL encode, I was actually able to spot the CU-boundry boxes issue without any zooming shenanigans, I even saw it on a "normal" frame that didnt have any crazy complexity to it, it was just a scenecut change... But in the end, if you are encoding for yourself, and if you are happy, good for you, but I woudlnt pass anything with this issue as a quality encode at this bandwidth, and it woudlnt (shoudlnt) pass QC.

Anyway, I think I already made most of the points I was trying to make, so Im gonna drop out here as it doesnt seem to be very productive as it seems like we are going circles. But I do agree that quality during normal playback is most important, and not stills, but that doesnt mean that stills are a bad tool for analysis, and SoL can be very much encoded at the same level of sharpness without those CU-boarders.

++ on this.

There's no plot of any metric that substitutes for eyeballs. There are lots of well known compression efficiency optimizations that reduce overall QP but improve subjective quality in double-blind testing. The same QP value can look flawless with complex foliage but terrible with text on a smooth gradient.

I cannot imagine any scenario where turning off chroma ME would actually improve subjective quality, whatever it does to metrics. Note that a lot of metrics (VMAF, SSIM, Y PSNR) completely ignore chroma, and I don't know how chroma and luma QP are combined to determine x265's logged frame QP value. Particularly with anime, poor chroma ME or QPs can cause some very unpleasant visual artifacts that are nearly invisible to metrics.

[HD MOVIE SOURCE]

Originally Posted by benwaggoner

++ on this.

There's no plot of any metric that substitutes for eyeballs. There are lots of well known compression efficiency optimizations that reduce overall QP but improve subjective quality in double-blind testing. The same QP value can look flawless with complex foliage but terrible with text on a smooth gradient.

I cannot imagine any scenario where turning off chroma ME would actually improve subjective quality, whatever it does to metrics. Note that a lot of metrics (VMAF, SSIM, Y PSNR) completely ignore chroma, and I don't know how chroma and luma QP are combined to determine x265's logged frame QP value. Particularly with anime, poor chroma ME or QPs can cause some very unpleasant visual artifacts that are nearly invisible to metrics.

I do think you're right. However, I was able to clear up some artifacts on an iframe by doing everything possible to lower QP values. I've tried using ssim, and psnr before and I felt that just getting lower QP values is where its at. Overall, on x265, I was able to lower the QP from around 8 to 7 from testing, and manipulating different settings. Visually, I was happy with it at 8, because the artifacts were absolutely impossible to see at full speed. So, what Im saying is, from my testing, that striving for lower QPs is a way to get s single screenshot to look closer to the prores 4:4:4 master. Getting lower QP values worked for me. I use my eyes to see if anything has visual artifacts, then I overkill it, by testing some more to find the lowest QP values. Without any other tools, it works for me. So, visually, what I see, screenshots to find blocking, lower QP to reduce or completely remove blocking is how I did the Sol levante encode. I think it's an encode that I couldn't get anymore out of. Unless there's some settings like hme, that are turned off, and need to be turned on because they are not part of any default settings.

With x264, is there content that would strongly benefit from chroma ME? Like animation? Maybe Big Buck Bunny, or is that too soft? I haven't got around to screen-capping my x264 encodes to really see what it does. I have just been doing x264 by eye and couldn't see the difference with chroma ME. x265 showed me that gaining a lower QP yielded better images, and I thought that using that method would be good for x264 also.
Visually, I think the x264 encode looks exceptional. I might encode Sol Levante using x264, just to see how it compares to x265. I would be using 1080p to keep it as close to a Blu-ray encode as possible. But I think that might give a good idea of what chroma ME does. If I could see what chroma ME actually does and it helped then I would just ignore the QP gain.

[HD MOVIE SOURCE]

Originally Posted by rwill

Are you using archive.org as some sort of personal file hosting service?

I have a google drive too, but I thought I'd put some encodes out there if anyone wanted to see what I've done. Its something I enjoy doing.

[benwaggoner]

Originally Posted by HD MOVIE SOURCE

I do think you're right. However, I was able to clear up some artifacts on an iframe by doing everything possible to lower QP values. I've tried using ssim, and psnr before and I felt that just getting lower QP values is where its at. Overall, on x265, I was able to lower the QP from around 8 to 7 from testing, and manipulating different settings. Visually, I was happy with it at 8, because the artifacts were absolutely impossible to see at full speed. So, what Im saying is, from my testing, that striving for lower QPs is a way to get s single screenshot to look closer to the prores 4:4:4 master. Getting lower QP values worked for me. I use my eyes to see if anything has visual artifacts, then I overkill it, by testing some more to find the lowest QP values. Without any other tools, it works for me. So, visually, what I see, screenshots to find blocking, lower QP to reduce or completely remove blocking is how I did the Sol levante encode. I think it's an encode that I couldn't get anymore out of. Unless there's some settings like hme, that are turned off, and need to be turned on because they are not part of any default settings.

Sol Levante seems to be a uniquely impossible source, and I've yet to see it look great at any distribution bitrate. A deep dive on why that is has been on my get-to-later list for several years now.

I concur that QP is a useful and underutilized metric, and use it myself for quality alarming. Rate Factor would be better yet (as it is QP with psychovisual complexity offset), but isn't logged by default.

With x264, is there content that would strongly benefit from chroma ME? Like animation? Maybe Big Buck Bunny, or is that too soft? I haven't got around to screen-capping my x264 encodes to really see what it does. I have just been doing x264 by eye and couldn't see the difference with chroma ME. x265 showed me that gaining a lower QP yielded better images, and I thought that using that method would be good for x264 also.
Visually, I think the x264 encode looks exceptional. I might encode Sol Levante using x264, just to see how it compares to x265. I would be using 1080p to keep it as close to a Blu-ray encode as possible. But I think that might give a good idea of what chroma ME does. If I could see what chroma ME actually does and it helped then I would just ignore the QP gain.

Chrome ME is useful in general, and most useful when luma and chroma move independently. This can happen with anime/cel animation and motion graphics a lot. I've seen it help with grain as well, to offset false motion vectors created by the grain.

[HD MOVIE SOURCE]

Originally Posted by benwaggoner

Sol Levante seems to be a uniquely impossible source, and I've yet to see it look great at any distribution bitrate. A deep dive on why that is has been on my get-to-later list for several years now.

I concur that QP is a useful and underutilized metric, and use it myself for quality alarming. Rate Factor would be better yet (as it is QP with psychovisual complexity offset), but isn't logged by default.


Chrome ME is useful in general, and most useful when luma and chroma move independently. This can happen with anime/cel animation and motion graphics a lot. I've seen it help with grain as well, to offset false motion vectors created by the grain.

Good to know. I'll rest some motion and see whats visible and whats not. Thanks.