"Visually lossless" encoding (or close to it) for UHD sources: experiences?

[jd17]

Hello doom9 community!

I am a new user in this forum...
However, I have been reading here for as long as I can remember.

I would like to pick your brains regarding reasonable presets for high quality HEVC encoding on UHD sources.


To give you a frame of reference for what I am looking for, this is how I encode and used to encode 1080p Blu-ray sources, based on my own trials and findings:

Originally, I used the following:
- x264 8bit
- Preset: Slow
- Tune: Film
- CRF: 16

- Audio: DTS-HD / TrueHD / Atmos passthrough

Recently, I switched to HEVC for 1080p sources:
- x265 10bit
- Preset: Medium
- Tune: Grain
- CRF: 17

- Audio: opus 1.1.5 (640kbit/s VBR for 8 channels, 480kbit/s VBR for 6 channels) for DTS-HD / TrueHD, passthrough for Atmos

My recent findings were the following:
- x265 8bit is useless in my eyes (at least for 1080p / 8bit sources), looks much worse than x264 8bit. Way too much banding.
- x265 @ no tune is too blurry for me.
- x265 10bit + Grain @ Medium @ CRF17 saves 20-30% in comparison to my old x264 8bit setting, while looking equal or better.
- opus obviously saves a lot of space, bitrate is even higher than needed for transparency (buffer for a clear conscience).


I know that the pro community here probably uses custom command line settings only, but I like the convenience of Handbrake and I have always been happy with the presets and tunes.

However, based on my settings for 1080p stuff, what do you think would be a good UHD (2160p / 10bit / BT.2020 / HDR10) source equivalent?
I encoded one of those LG or Samsung UHD demos with my usual HEVC settings - it looked perfect to me and saved some space too... But it also took ages (encoding speed was 1.7 fps average) and I was just wondering if I can squeeze either a few more fps or a higher saving out of it, while still being "visually lossless" or close to it.

Ideally I should do my own trials again, I know that of course.
But that seems much more difficult on my non-UHD PC, I'd always have to go to the TV and side-by-side comparisons seem very difficult too...

So I thought I'd ask you guys, maybe you are already there and have some good advice for me?

Thank you very much in advance!

[jd17]

Is there something wrong with my question or am I just too early in asking it?

[Gser]

This has some good hints.
https://forum.doom9.org/showthread.php?t=172458

Tune grain takes care of most of it, except for qcomp and deblock settings. Personally I use --no-strong-intra-smoothing on higher CRFs, you'll just have to encode and see if it makes a difference for you.

[Motenai Yoda]

first, what version of x265 are you comparing to x264? as the last lambda2 patch greatly improve low detail retention
second you are using preset medium which disable psy-rdoq enhancement too

btw iirc opus max bitrate is about 510kbps, and imho it's just overkill

[jd17]

Originally Posted by Gser

This has some good hints.
https://forum.doom9.org/showthread.php?t=172458

Tune grain takes care of most of it, except for qcomp and deblock settings. Personally I use --no-strong-intra-smoothing on higher CRFs, you'll just have to encode and see if it makes a difference for you.

As far as I see it, this thread is mainly about 1080p sources, or am I wrong? I thought things might be different for 2160p? Are they not?

Apart from that, I did read a bit in that thread before but the post from "sneaker_ger"...
https://forum.doom9.org/showthread.p...01#post1733901
...eventually made me decide against using those settings.
While that custom film tune adds some detail, it also introduces a lot of artifacts.
To me grain seems much closer to the source and I do have more detail due to using CRF 17 instead of 19.5-20 in the example.

The whole recommendation is also based on the slower preset, which is just unbearably slow on my i5-7500.
I did however compare x265 10bit CRF17 grain at presets fast, medium, slow and slower and there was no significant gain in quality of slow or slower over medium in my eyes.

[jd17]

Originally Posted by Motenai Yoda

first, what version of x265 are you comparing to x264? as the last lambda2 patch greatly improve low detail retention

I am using x265 10bit 2.3 from this post:
https://forum.handbrake.fr/viewtopic...=10bit#p169744

second you are using preset medium which disable psy-rdoq enhancement too

Yes, I kind of answered why I use medium in the previous post...
Should I manually enable psy-rdoq for medium?

btw iirc opus max bitrate is about 510kbps, and imho it's just overkill

510kbit/s is maximum for stereo and yes, overkill for stereo of course.
opus recommends up to 450kbit/s for 8 channels, so I just added some bitrate for my clear conscience.
The files are still small and setting 640kbit/s VBR for 8 channels usually ends up at around 530kbit/s average (range between 470kbit/s and 620kbit/s until now).

[jd17]

Originally Posted by Motenai Yoda

as the last lambda2 patch greatly improve low detail retention

I just googled a bit and saw that now there is x265 2.4 - I assume you are referring to that update?
Do you know where I can get the according libx265_main10.dll needed for Handbrake in Windows?
Unfortunately, the Handbrake team has not released that yet...

[davidsama]

http://www.msystem.waw.pl/x265/

[jd17]

Thank you so much!

I'll try to conduct some comparisons between 2.3 and 2.4...

Due to the lack of responses I assume there are not many 2160p experiences yet..?

[Winston_Smith_101]

For 1080p sources I use these settings and the output is visually nearly lossless to my eyes: "--crf 18 --preset veryslow --output-depth 10 --rskip --no-sao" The Options are explained here: http://x265.readthedocs.io/en/default/cli.html

The "--no-sao" Switch makes a huge difference. The Output gets much sharper. It kills the "blurryness" you mentioned in your first post. -> Try it! Personally I dont use "tune grain", because the files get so much bigger with this option and with the "--no-sao" switch the output quality is far good enough.

I assume that for 2160p sources --crf 19 or 20 could be enough.

I hate to say it, but your Intel Core i5-7500 is not fast enough for hq x265 encoding. YOu can do it, but it takes ages. You need a much faster High End CPU for this.

[AMED]

Is there much of the bitrate bump by using the --no-soa with your current settings?

[Winston_Smith_101]

Originally Posted by AMED

Is there much of the bitrate bump by using the --no-soa with your current settings?

I can't see any significant increase in bitrate with "--no-sao".

[jd17]

Originally Posted by Winston_Smith_101

For 1080p sources I use these settings and the output is visually nearly lossless to my eyes: "--crf 18 --preset veryslow --output-depth 10 --rskip --no-sao" The Options are explained here: http://x265.readthedocs.io/en/default/cli.html

The "--no-sao" Switch makes a huge difference. The Output gets much sharper. It kills the "blurryness" you mentioned in your first post. -> Try it! Personally I dont use "tune grain", because the files get so much bigger with this option and with the "--no-sao" switch the output quality is far good enough.

I assume that for 2160p sources --crf 19 or 20 could be enough.

Thanks for your help!
Although I doubt that your settings are enough for me on 1080p sources (even tune grain is already worse compared to x264 film on sources with grain), I think just using "--no-sao" seems like a very good alternative for 2160p.
I will definitely do some comparisons (for 1080p too, of course).

I just wonder why you would use the "veryslow" preset, nobody seems to go slower than "slower"...
Do you actually see a visual benefit of veryslow over slower, slow or medium? ...Or a reasonable file size reduction?

I hate to say it, but your Intel Core i5-7500 is not fast enough for hq x265 encoding. YOu can do it, but it takes ages. You need a much faster High End CPU for this.

Well, yes and no.
I bought the i5-7500 because it is one of the most efficient CPUs out there. Even though it's obviously slower than an i7-7700 for instance, the efficiency of the i7 is worse (i.e. the power consumption over encoding time is higher).

I also found that more and more threads only get you diminishing returns.

I'm perfectly happy with the 1080p encoding speed of x265, but 2160p is definitely a different beast.
I assume your CPU is much more powerful, what encoding speed do you get on 2160p / 10bit / BT.2020 / HDR10 sources when you use x265 10bit, preset medium, tune film and CRF17?

[Winston_Smith_101]

Originally Posted by jd17

Although I doubt that your settings are enough for me on 1080p sources (even tune grain is already worse compared to x264 film on sources with grain), I think just using "--no-sao" seems like a very good alternative for 2160p.
I will definitely do some comparisons (for 1080p too, of course).

These settings do not produce REALLY lossless results, but are very pleasing to my eyes, more that necessary.
Crf 17 results are more detailed, but not necessary for me. They are only much larger in filesize. Without direct comparison I see no obvious losses.

Originally Posted by jd17

I just wonder why you would use the "veryslow" preset, nobody seems to go slower than "slower"...
Do you actually see a visual benefit of veryslow over slower, slow or medium? ...Or a reasonable file size reduction?

I use veryslow because of the file size reduction. Maybe this is not really necessary, but I'm not in a hurry. My processors are fast enough. I use x265 because I like high quality movies without excessive bitrates. I would like to use the full potential of the codec. Ok, the "placebo" settings are to slow for my machine. ;-)

My computer has 2 Intel Xeon e5-2683 v3 CPUs each with 14 cores. If I encode 4 1080p movies at the same time x265 uses all cores very well. Sorry, I have so far no experience with 2160p encodes.

[jd17]

Originally Posted by Winston_Smith_101

The "--no-sao" Switch makes a huge difference. The Output gets much sharper. It kills the "blurryness" you mentioned in your first post. -> Try it! Personally I dont use "tune grain", because the files get so much bigger with this option and with the "--no-sao" switch the output quality is far good enough.

You Sir, are a genius.

Simply adding --no-sao makes such a big difference!
However, the bitrate bump is very minor and encoding speed also barely suffers.

I created a series of comparison screenshots last night, I'll share them when I get home today.

Essentially it's obviously not as close to the source as tune grain, but so much better than regular tune=none without adjustments.
It really seems like the best "bang for the buck".

I cannot understand why sao is not disabled by default.
Is it essential at low bitrates?


I updated my personal encoding presets accordingly.
https://docs.google.com/spreadsheets...it?usp=sharing

[jd17]

Originally Posted by Winston_Smith_101

My computer has 2 Intel Xeon e5-2683 v3 CPUs each with 14 cores. If I encode 4 1080p movies at the same time x265 uses all cores very well.

That is just insanity!
So you have 56 threads in one machine?!

I have to admit that I was referring to x264 regarding diminishing returns of more cores / threads.
The performance gains may very well be much better on x265, I don't know.

Does that in return mean that x265 utilizes 14 threads well for one movie?

[jd17]

I used avidemux to go to the exact frame, made a screenshot with the print key and saved the file as .tif.
Obviously I had to convert the image to .jpg for the upload, I used paint.net and moved the slider to 100%.

The screenshot is a B-frame from "Captain America: Civil War".
5 minute source sample is 812 MB.

1. x265 10bit lossless. tif file size is 1.36 MB.
http://i.imgur.com/KvlXsfT.jpg
(I created a 10bit lossless encode to avoid 10bit to 8bit dithering noise difference in the image comparison.)

2. x265 10bit CRF17 medium grain, v2.4. tif file size is 1.26 MB.
http://i.imgur.com/Gy5id4Z.jpg
Video is 267 MB.

3. x265 10bit CRF17 medium grain, v2.3. tif file size is 1.25 MB.
http://i.imgur.com/QSdrHUW.jpg
Video is 238 MB.

4. x265 10bit CRF17 medium none, no-sao, v2.4. tif file size is 1.21 MB.
http://i.imgur.com/SHSqgjJ.jpg
Video is 108 MB.

5. x265 10bit CRF17 medium none, v2.4. tif file size is 1.07 MB.
http://i.imgur.com/C7GrH4D.jpg
Video is 99 MB.

6. x265 10bit CRF17 medium none, v2.3. tif file size is 1.07 MB.
http://i.imgur.com/MKZjAbL.jpg
Video is 89 MB.

[Winston_Smith_101]

Originally Posted by jd17

That is just insanity!
So you have 56 threads in one machine?! Does that in return mean that x265 utilizes 14 threads well for one movie?

Thank you. x265 utilizes 14 threads very well. It can utilize more than 14 threads! I encode 4 1080p movies at the same time with 14 threads each, because these utilize the 2 CPUs a little bit better than just 2 movies with 28 threads each. But the encoding speed with just 2 movies and 28 threads each is per movie somewhat faster. x265 with one 1080p movie utilizes 28 threads on average 70-100%, but 2 1080p movies with 14 threads each 90-100%. So I decided to encode 4 movies at the same time. :-)

My screenshot is in the attchment.

Originally Posted by jd17

I cannot understand why sao is not disabled by default.

A very good question, but I dont know the answer. It is also a mystery to me. Because of sao I did't like x265 at the beginning.

[jd17]

Originally Posted by Gser

Tune grain takes care of most of it, except for qcomp and deblock settings.

I added --qcomp 0.8 and --deblock -2:-2 to tune grain, but the result looks a bit worse than standard tune grain to me.
However, there is a bit of a file size reduction.

Personally I use --no-strong-intra-smoothing on higher CRFs, you'll just have to encode and see if it makes a difference for you.

I tried that as well.
While the difference is extremely subtle when added to tune grain, adding --no-strong-intra-smoothing to "tune none" degrades picture quality in my opinion.
It introduces a bit of banding that is not there in the source.

[Gser]

Originally Posted by jd17

It introduces a bit of banding that is not there in the source.

Even in 10-bit?