Let's Talk about Blu-Ray Transparent x265 10bit Encodes

[StormMeows]

Hey guys,

I have been encoding a lot of live action 1080p blu-rays using really high quality settings in x264 8-bit. Since I am going to be storing these high quality blu-ray backups for a long time, I want to "future proof" and use x265 10bit to save some extra space as well.

When I am aiming towards transparency to the source with x265, will the preset slower be enough, or should I go "veryslow?" I use veryslow for x264 with some of the placebo settings (pretty much everything, but I don't use tesa (I use umh instead). For samples, I will test each CRF value ranging from 15-25 and interleave the source to the various CRF test samples. From there, I pick a CRF that looks close to the source and that's my typical CRF.

I did a lot of test samples and the x265 definitely saves some space but it looks pretty similar to my x264 8 bit encodes and the encoding time is at least 5x longer on the x265 encode. I'm not sure that's really worth it. So I could do 5 movies per day average with x264, even at the slowest placebo settings. Whereas with x265, I can do only 1 a day, even on a very fast dedicated encoding machine. Wow.

Here are some of the settings I am thinking for x265 10bit encodes to maintain transparency to the source:

--preset slower (or very slow if you recommend it)
--no sao
--no cutree
--psy RD .9-2.0
-AQ mode 3 or 4
--bframes16
--rc-lookahead 60
--high tier
--deblock -3:-3 or -2:-2 (I use -3:-3 for live action x264)
--r-skip 0

Please let me know your thoughts on these settings and if there is anything else I should add to maintain transparency to the source. Keep in mind that when I sample using 2 pass encoding (based off of the bitrate CRF gave me), I compare various Psy-Rd values, along with qcomp, aqstrength, and aq mode (I typically just use 3 for x264 so I may do the same or mode 4 with x265).

Thank you guys!

[benwaggoner]

If you are coming from 8-bit source and not doing any scaling, there's no huge advantage to encoding in 10-bit HEVC, and you'll save some encoding time.

Your settings don't make a lot of sense to me. Content that can benefit from --bframes 16 would also get a lot of value from --cutree, for example.

x265 can get away with higher --crf than x264 in some cases. 15 is definitely overkill. 18 is a good starting point to tune from; I think --crf 17 is about the lowest I've found made sense for 1080p content.

There is something off with your file size/bitrate math. file size is ABR * duration, so if the bitrates are close the file sizes will be as well. An 8x difference is crazy; maybe there was a bit/byte conversion error somewhere?

If you are trying to do quality-per-bit comparisons, you're better off using 2-pass VBR encoding using the same bitrate value, so bitrate is apples-to-apples.

I don't think --aq-mode 4 has been shown to be optimal for pretty much any content, and --aq-mode 3 can help reduce banding/blocking near black, but can really increase bitrate for a given CRF. You might find using it and raising CRF by 1-2 could net out. But if you're having those sorts of issues, just using 10-bit is probably the simplest. The fancy new AQ mode is --hevc-aq, but that has not been demonstrated to have a reliable quality improvement over good old --aq-mode 2.

You don't need to lower --deblock as much in x265 as with x264.

--high-tier is probably not needed, and will reduce compatibility. I'd probably use --idc-level 4.1, which allows for --vbv-maxrate and --vbv-bufsize to be 20000, which should be sufficient for 1080p24 Blu-ray sources.

--rskip 2 can possibly help with grainy content, but if you're dealing with that you'll want a higher --psy-rd and --psy-rdoq. --rskip 0 is quite a lot slower than either 1 or 2.

A little --nr-inter can help if you've got grain issues, around 100-150 is generally transparent.

Honestly, I'd start with just --preset slower with just basic settings, and then introduce more advanced parameters from there. The more weird combinations you introduce, the farther away you'll be from the most tested x265 combinations, and things can get unpredictible.

[StormMeows]

If you are coming from 8-bit source and not doing any scaling, there's no huge advantage to encoding in 10-bit HEVC, and you'll save some encoding time.

Your settings don't make a lot of sense to me. Content that can benefit from --bframes 16 would also get a lot of value from --cutree, for example.

x265 can get away with higher --crf than x264 in some cases. 15 is definitely overkill. 18 is a good starting point to tune from; I think --crf 17 is about the lowest I've found made sense for 1080p content.

There is something off with your file size/bitrate math. file size is ABR * duration, so if the bitrates are close the file sizes will be as well. An 8x difference is crazy; maybe there was a bit/byte conversion error somewhere?

If you are trying to do quality-per-bit comparisons, you're better off using 2-pass VBR encoding using the same bitrate value, so bitrate is apples-to-apples.

I don't think --aq-mode 4 has been shown to be optimal for pretty much any content, and --aq-mode 3 can help reduce banding/blocking near black, but can really increase bitrate for a given CRF. You might find using it and raising CRF by 1-2 could net out. But if you're having those sorts of issues, just using 10-bit is probably the simplest. The fancy new AQ mode is --hevc-aq, but that has not been demonstrated to have a reliable quality improvement over good old --aq-mode 2.

You don't need to lower --deblock as much in x265 as with x264.

--high-tier is probably not needed, and will reduce compatibility. I'd probably use --idc-level 4.1, which allows for --vbv-maxrate and --vbv-bufsize to be 20000, which should be sufficient for 1080p24 Blu-ray sources.

--rskip 2 can possibly help with grainy content, but if you're dealing with that you'll want a higher --psy-rd and --psy-rdoq. --rskip 0 is quite a lot slower than either 1 or 2.

A little --nr-inter can help if you've got grain issues, around 100-150 is generally transparent.

Honestly, I'd start with just --preset slower with just basic settings, and then introduce more advanced parameters from there. The more weird combinations you introduce, the farther away you'll be from the most tested x265 combinations, and things can get unpredictible.

Hey thanks for your thorough response. I really appreciate it. My conversion was wrong. I will update my post. When I tested in Staxrip, I forgot to take off my lossless track, lol.

I typically do not use a CRF 15 when doing x264 8 bit encodes. I usually test with a CRF from 15 to 22 because I know the CRF isn't going to be transparent to the source, whereas the CRF will be. This let's me see the quality differences. I go with the hoghest CRF value that provides a transparent encode. I typically see a bit of grain shift but usually CRF17 is pretty transparent with x264 8 bit. I'm sure it will depend on the movie and scene. I also consider the file size outcome as well. So I take the bitrate from the CRF value I choose and test it with 2 pass. 2 pass also allows me to test various qcomps, aq strength, aq mode, and psy-rd values. From your experience, do i need to test those settings for each movie? My tests can kind of get out of hand but I'm not one of those encoders who using zoning throughout the movie. This is why I need to use good settings and a lower CRF to be transparent to the source throughout the movie.

The main reason why I thought it would be a good idea to switch to x265 10 bit over x264 8 bit is 1)compression at the same bitrate would be better and they would look the same 2)10 bit typically introduces less banding 3)it may be more future proof. After testing my settings yesterday the encode time was crazy for the x265 10 bit encode. On my 32 thread AMD 5950x encoding rig, it was going to take 24-26 hours to do 1 movie. In comparison, it takes me 5 to 6 hours typically to do an x264 8 bit encode on PLACEBO. So that means even if I used placebo on the x264, I could backup 5 movies compared to 1 movie per day. I have a TON of bluray media so speed does matter somewhat but I don't want to sacrifice quality. I am looking for around a 35% reduction in size to the original. If I can't achieve that percentage for a specific movie, I just keep it as a Remux. For instance, the movie 8 mile is only like 17GB. If I encoded it with transparency, I only save a few GBs. Not worth it on that movie but for most movies I can usually shave off around 40% and keep the lossless audio track.

Also, I typically use 16 bframes because it doesn't slow my encode down too much, aids in compression, and I don't feel like counting how many bframes is needed for every movie using the logs. (Seeing when the b frames drop below 1%).

If you have any tips/advice on what route I should go, that would be great. You have been very helpful. My main concern is should I go x265 10 bit or stick to x264 8 bit for 1080p live action encodes. I realize when I go to do my 4Ks I will NEED to do x265 10 bit with HDR but I am not there yet. Also, if you think sticking to x264 makes sense, if you could please share some settings you typically use, that would be great. I want the encode to be transparent to the bluray source. So if the bluray looks really grainy and crap, then I want the same for my encode. I don't want to fix anything on the bluray, just reduce the size a bit per movie. Also, if you could go over a good plan for testing aq-strength, ip ratio/pb ratio, qcomp, psy-rd, that would be great. For instance I typically use a bit higher of a q comp for grainy movies and lower the aq strength. Also, it seems like with x265 and x264 aq-mode 3 is pretty safe to use on most movies. Dark scenes typically are the most problematic so thats why I chose aqmode 3.

EDIT: Did some more testing from what you said and just did "slower x265 10 bit" and it takes about 8:30 - 9hrs. Placebo settings maxed out settings with x264 8 bit only takes 5-6hrs for the same movie. That doesn't seem to be worth it, right? Thanks

Sorry for the long response. I have a lot on my mind and ant help would be greatly appreciated. Thanks and stay safe!

[ReinerSchweinlin]

I ahve been doing some tests with high b-frame numbers recently. Only for CG-Cartoons from very good sources it made a difference in file size. At the same time, a lot of playback devices started to struggle with playback with b-frames 16...

I settled for 8 bframes for CG Anime / Cartoon and most other stuff, max number was 6. Encoding time is faster, the difference in file size is very little and playback devices don´t struggle :)

[StormMeows]

I ahve been doing some tests with high b-frame numbers recently. Only for CG-Cartoons from very good sources it made a difference in file size. At the same time, a lot of playback devices started to struggle with playback with b-frames 16...

I settled for 8 bframes for CG Anime / Cartoon and most other stuff, max number was 6. Encoding time is faster, the difference in file size is very little and playback devices don´t struggle :)

I haven't really thought about compatibility like that. I will have to test some of my files that are bframes 16 since I mainly use that. If you wanted to get technical you can check out the logs on your test encodes and see how many bframes are "needed" per each encode. I may end up doing that if the efficiency isn't a bonus with 16. Just check the log and see how many b-frames are over 1%. That's the number to use for the encode.

[ReinerSchweinlin]

I didn´t make a spreadsheet or statistic, but looked at the end-statistics of the b-frame distirbution frome time to time. As far as I remember, the distribution peak of them moved from around 4 for real-life high-action movies to around 7 with cg-created cartoons with "classic" looks (a lot of uniform areas, line drawings..).. The deviation to the right (more b-frames) fell of more quickly, the higher the b-frame count. b-frames of 10 or more rarely got more than a few percent, in most cases above 10 b-frames there was around 1% or 2%... So I figured - for my usecases - it´s not really worth it.
I found other means to "squeeze" out some efficiency with - for me - acceptable quality losses (slight noise reduction with external filters or in the x265 settings or deduplication in cartoons..) while still getting reasonable encode times (in my case it would have been kiond of silly to spend 300% encoding time and pay a lot of electricity only to save 4% space - I´d rather simply get more HD space :) ). But of course, sometimes, it´s just a "sport" - and in streaming with limited bandwith, it´s always nice to see where the limits are...

[Boulder]

The noisier the source is, the more B-frames the encoder seems to use. It's quite strange as I would expect a clean source with less difficulties in analyzing motion to utilize them more. I've seen quite many encodes having 20-30% of 8 consecutive B-frames, hence I nowadays use --bframes 10.

[StormMeows]

The noisier the source is, the more B-frames the encoder seems to use. It's quite strange as I would expect a clean source with less difficulties in analyzing motion to utilize them more. I've seen quite many encodes having 20-30% of 8 consecutive B-frames, hence I nowadays use --bframes 10.

Thanks for the info! I figure might as well just be safe and use bframes 16 on everything. I haven't noticed any playback issues yet, but you're right, 10 is also safe to use as well. I still haven't heard back from benwaggor, can you look over my x264 8 bit questions and see if you can assist at all? Greatly appreciated. thanks

[Boulder]

x264 (8-bit) is good enough for archiving if your eyes tell you so ;)

I just find it strange that it takes 24 hours to encode a normal movie at --preset slower with your CPU. --rskip 2 --rskip-edge-threshold 2 --no-amp --limit-refs 3 --ref 4 would probably help a lot though, added with --no-sao and --deblock -1:-1. CRF 18 is what I'd test first.

EDIT: --ctu 32 would probably help utilizing your CPU more but there may be a (probably negligible) quality loss at 1080p.

[StormMeows]

x264 (8-bit) is good enough for archiving if your eyes tell you so ;)

I just find it strange that it takes 24 hours to encode a normal movie at --preset slower with your CPU. --rskip 2 --rskip-edge-threshold 2 --no-amp --limit-refs 3 --ref 4 would probably help a lot though, added with --no-sao and --deblock -1:-1. CRF 18 is what I'd test first.

EDIT: --ctu 32 would probably help utilizing your CPU more but there may be a (probably negligible) quality loss at 1080p.

Thank you, I was doing "very slow" preset with probably some placebo tweaks prior. If I just do default "slower" preset then I think it was around 9-10 hours if I remember correctly. That would still be 3-3.5x times slower than my x264 8-bit 1080p bluray encodes that use very slow presets.

[benwaggoner]

The noisier the source is, the more B-frames the encoder seems to use. It's quite strange as I would expect a clean source with less difficulties in analyzing motion to utilize them more. I've seen quite many encodes having 20-30% of 8 consecutive B-frames, hence I nowadays use --bframes 10.
I don't know how well-tested >8 b-frames are, since they aren't in any of the x265 presets except when using --tune-animation. More than 10 would be really rare in practice.

[StormMeows]

I don't know how well-tested >8 b-frames are, since they aren't in any of the x265 presets except when using --tune-animation. More than 10 would be really rare in practice.

From my understanding is that b-frames 16 for x264 and x265 are "fine" but just might be overkill. I use b-frames 16 on x264 and x265 so I don't have to count the b-frames on the log for each movie. Sometimes bframes 16 helps with compression as well from what I have read.

[Boulder]

On an Amlogic device (like my ODroid N2), 10 B-frames were an issue with one older CoreELEC (OS for Kodi) build. It was due to the player dropping frames unnecessarily as I think there were some delay calculations which tripped with values higher than 8. I reported it to the CE devs and it was promptly fixed by a patch. So yes, there always might be issues.

[StormMeows]

On an Amlogic device (like my ODroid N2), 10 B-frames were an issue with one older CoreELEC (OS for Kodi) build. It was due to the player dropping frames unnecessarily as I think there were some delay calculations which tripped with values higher than 8. I reported it to the CE devs and it was promptly fixed by a patch. So yes, there always might be issues.

Oh wow, I use an nvidia shield pro and it doesn't seem that 16 bframes has been causing any issues. I may want to just count each bframe then in the log to maximize the bframes for each movie. I only want to count the bframes value until it the % is less than 1% right? Thanks

[Boulder]

I think you should compare values like 8, 10 and 16 and see what the actual difference is regarding the filesize and encoding speed. 16 B-frames could be much slower to encode than 8 or 10 but the difference in filesize still very small. Most of the time 8 is enough.

[StormMeows]

I think you should compare values like 8, 10 and 16 and see what the actual difference is regarding the filesize and encoding speed. 16 B-frames could be much slower to encode than 8 or 10 but the difference in filesize still very small. Most of the time 8 is enough.

Most definitely! I built a dedicated machine for encoding (5950x) so the slowdown on the 16 hasn't been too dramatic. A lot of the articles I read state to just leave bframes at 16 if you don't want to count the correct number for each movie. I think either is fine. As you mentioned, bframes 16 may save me a tiny bit of file size and the encode shouldn't be any worse than a lower b-frames amount.

[ReinerSchweinlin]

The noisier the source is, the more B-frames the encoder seems to use. It's quite strange as I would expect a clean source with less difficulties in analyzing motion to utilize them more. I've seen quite many encodes having 20-30% of 8 consecutive B-frames, hence I nowadays use --bframes 10.
That´s interesting. I made other experiences, but of course one can be mistaken - I´ll have to investigate this if I have time...