x265 HEVC Encoder

[nghiabeo20]

The OS scheduler does do that, unless you set the process to use only specific cores (=CPU affinity). But as I and Atak_Snajpera said, set the process priority lower (idle or just below normal) and let the encoder use whatever resources are available to it after other processes.

Why don't you try runing x265 in idle priority? What you do is not optimal.

It's a laptop, so I still don't want 90+% usage for an extended period of time. I'm saving money for a Ryzen rig anyway.

Windows: https://blogs.msdn.microsoft.com/santhoshonline/2011/11/24/how-to-launch-a-process-with-cpu-affinity-set/

Linux: man taskset

OS X: You're f*cked.

:p Yes i'm a fucked TM user here.

[Forteen88]

You could try --frame-threads 1 to limit encoding to one frame at a time.Yeah, I'd do this if I were the person you answered to, since this also increases the picture-quality of the encode.

[redbtn]

At those rates you may very well get better results with AVC, to be honest.I've done tests, 1080p with bitrate 22mb, x264 10bit (placebo, subme 11, bframes 8, etc) and x265 10bit (slow, no-rect, subme 4, bframes 8 and some other tweaks). In the end I get almost the same encoding speed. And x265 looks much better. I don't know why many people say that at high bitrate x264 beats x265.

[Atak_Snajpera]

I've done tests, 1080p with bitrate 22mb, x264 10bit (placebo, subme 11, bframes 8, etc) and x265 10bit (slow, no-rect, subme 4, bframes 8 and some other tweaks). In the end I get almost the same encoding speed. And x265 looks much better. I don't know why many people say that at high bitrate x264 beats x265.

BS. 22 Mbps is insanelly high for 1080p. That's basicaly a blu-ray teoritory. You are lieing saying that x265 looks MUCH BETTER.

[redbtn]

BS. 22 Mbps is insanelly high for 1080p. That's basicaly a blu-ray teoritory. You are lieing saying that x265 looks MUCH BETTER.

I know that 22 is very high for 1080p, I did it specially for test. After encode I upscaled FHD to UHD and compared them. I don't say that x265 MUCH better, but it significantly better at preserving small details. And I can definitely say it's not worse anyway.

[Blue_MiSfit]

It's definitely not impossible that x265 would produce better results than x264 at 22 Mbps for 1080p. I think it's fair to say that for MOST content the difference would be quite negligible, especially at the same encoding speed. Even if you can get a small improvement when encoding several times slower, is it really worth it?

In many use cases, probably not. But for some? Absolutely.

[redbtn]

It's definitely not impossible that x265 would produce better results than x264 at 22 Mbps for 1080p. I think it's fair to say that for MOST content the difference would be quite negligible, especially at the same encoding speed. Even if you can get a small improvement when encoding several times slower, is it really worth it?

In many use cases, probably not. But for some? Absolutely.Yes, you are right, it absolutely impossible notice the difference at 22mb between x264 and x265 just by watching video. Like I said, I upscaled screens to 4K for noticing difference.
But IMO, if x265 better at low bitrates and better at high bitrates, I don't see why I should use x264 at any cases.

[Magik Mark]

I'm getting this warning. I'm trying to test the new frame duplication switch:

x265 [warning]: Frame-duplication require NAL HRD and VBV parameters. Disabling frame duplication

How do I address this?

[microchip8]

I'm getting this warning. I'm trying to test the new frame duplication switch:

x265 [warning]: Frame-duplication require NAL HRD and VBV parameters. Disabling frame duplication

How do I address this?

--hrd --vbv-maxrate <value> --vbv-bufsize <value>

[Magik Mark]

--hrd --vbv-maxrate <value> --vbv-bufsize <value>

What value shall I use?

[microchip8]

What value shall I use?

This depends. maxrate sets a specific maximum bitrate ceiling while buffer size sets a buffer, which usually should be higher than maxrate

[Majorlag]

What value shall I use?

For the values, you can use anything you want, but I would recommend using values to meet a certain Tier and Level so you can get Hardware accelerated decoding. You can see those at https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding, under Tiers and levels section.

I usually set bufsize and maxrate as the same. I cannot find anywhere that shows how high bufsize can be set to to maintain DXVA hardware decoding, so I use the following for all my encodes for x265.

480p:--vbv-bufsize 12000 --vbv-maxrate 12000
1080p:--vbv-bufsize 20000 --vbv-maxrate 20000
2160p:--vbv-bufsize 25000 --vbv-maxrate 25000

These have all resulted in DXVA compatible files.

[aymanalz]

With my settings above and HME turned on, I can shave ~100 MiB of an encode. Encoded Blade Runner once with and once without HME. The result was 100 MiB in size reduction with better subjective quality when HME is on

What was the total filesize, from which you shaved off 100 Mb? I mean, what was the percentage reduction in bitrate? Also, what was the video resolution?

I'm trying to figure out if HME is worthwhile for me, given that the encoding is significantly slower.

[jlpsvk]

For the values, you can use anything you want, but I would recommend using values to meet a certain Tier and Level so you can get Hardware accelerated decoding. You can see those at https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding, under Tiers and levels section.

I usually set bufsize and maxrate as the same. I cannot find anywhere that shows how high bufsize can be set to to maintain DXVA hardware decoding, so I use the following for all my encodes for x265.

480p:--vbv-bufsize 12000 --vbv-maxrate 12000
1080p:--vbv-bufsize 20000 --vbv-maxrate 20000
2160p:--vbv-bufsize 25000 --vbv-maxrate 25000

These have all resulted in DXVA compatible files.

encoding 2160p with 25000??? Level 5.1 @ High10 allows 160000 for vbv

[excellentswordfight]

encoding 2160p with 25000??? Level 5.1 @ High10 allows 160000 for vbv
And main teir is 40000.

If the only concern is level compatibility then I usually just set --level in the command line and x265 will automatically set the vbv and limit max ref frames etc to be compliant. But afaik some decoders will not handle such big bufsize even though they specification says that it supports a specific level, so there is viability to set custom (lower) values to ensure playback on an targeted device. But all my hevc level 5.1 devices have had no playback issues with encodes using the values in high teir though.

[microchip8]

What was the total filesize, from which you shaved off 100 Mb? I mean, what was the percentage reduction in bitrate? Also, what was the video resolution?

I'm trying to figure out if HME is worthwhile for me, given that the encoding is significantly slower.

Resolution was 1080p and file size with HME on was 3.7 GiB while with HME off was a bit over 3.8 GiB. Slowdown, IIRC, was about 5-6%

In the end, I recommend doing your own tests and see if it's worthy :)

[Majorlag]

encoding 2160p with 25000??? Level 5.1 @ High10 allows 160000 for vbv

I guess I could expand my reasoning as for why I would use those settings, since I do only encodes of 29.97fps video, then according to the graph I only need level 5.0, (4,096x2,160@30.0). I could set 5.1 spec, which allows up to 60fps, I don't encode 60fps, but I always use 10bit. I also use the 4k Ultra Blu-ray specifications at http://www.blu-raydisc.com/assets/Downloadablefile/BD-ROM_Part3_V3.0_WhitePaper_150724.pdf which says Main10 (which I understand to be Main10bit) or High (which I understand to be High8bit) or 5.1 Level max settings.

So I try to make sure I am as compatible with all my encodes as I can possible be. I hope I read the specs correctly.

[benwaggoner]

And main teir is 40000.

If the only concern is level compatibility then I usually just set --level in the command line and x265 will automatically set the vbv and limit max ref frames etc to be compliant. But afaik some decoders will not handle such big bufsize even though they specification says that it supports a specific level, so there is viability to set custom (lower) values to ensure playback on an targeted device. But all my hevc level 5.1 devices have had no playback issues with encodes using the values in high teir though.
Lots of real-world devices support > main tier but not all of high tier. UHD Blu-ray supports 80 Mbps Level 5.1, for example. For typical long-GOP distribution encoding, the Main Tier limits are fine for transparent quality with most sources.

[excellentswordfight]

Lots of real-world devices support > main tier but not all of high tier. UHD Blu-ray supports 80 Mbps Level 5.1, for example. For typical long-GOP distribution encoding, the Main Tier limits are fine for transparent quality with most sources.

Table 2-3–Specification of BD-ROM Primary video streams: Primary VideoCodec: HEVC (Main 10, High Tier,Level 5.1) Max. bitrate 100Mbps

http://www.blu-raydisc.com/assets/Downloadablefile/BD-ROM_Part3_V3.0_WhitePaper_150724.pdf

The different disctypes will then have different maximum ts bitrates though, I guess that the discs that allow for 81.7 Mbps are the most common.

But when it comes to bufsize, is it common that that devices has a smaller buffer then the allowed maxrate for the supported level? If I remember it correctly HD blurays specified a smaller one (30 vs 40Mbps), and the same was true for PS3 and xbox360 compliant encodings.

If I read the whitepaper correctly, it looks like this is the bufsize specificaiton for UHD-Bluray for the HEVC stream:

"DPB1 Decoded picture buffer for HEVC stream: 93312000 [bytes]"

[cap5lock]

Other than using external film grain filter,
Is there any x265 parameters to ignore bluray film grain/noise to further reduce size ?

[RanmaCanada]

Other than using external film grain filter,
Is there any x265 parameters to ignore bluray film grain/noise to further reduce size ?
Anything with high grain should only be encoded in x264 (or just remuxed) as x265 still can not handle it anywhere near as well. I believe the last time someone tested it, they were doing the original Alien, and the re-encode came out larger than the source.

Unless it's magically gotten better since then.

[Blue_MiSfit]

Depends on the intended use case. If you're going for total transparency you might have a harder time. If you're trying to get down to 10-15 Mbps for 1080p you'll probably get good results with the grain tuning and some nice slow settings, but it might be hard to get a reduction if you're coming from a BluRay source.

Getting down to a lower bitrate without blowing up and maintaining some of the grain texture is usually possible, but always difficult. Grain is hard no matter what.

I'm hoping the film grain synthesis tools in AV1 get explored soon - I've always thought this was a great idea.

I also wonder if the new MPEG-5 LC-EVC / enhancement layer systems could be used to encode a cleaner version of the video and then grain synthesis information as an enhancement layer / noise model, kind of like V-Nova Perseus but tuned for grain.

[Zebulon84]

I'm hoping the film grain synthesis tools in AV1 get explored soon - I've always thought this was a great idea.
It seams Netflix is working on an AV1 film grain synthesis : AOMedia 2019 Research Symposium - 04 : Netflix (https://youtu.be/M1vwnI0vbMI?t=147).

[excellentswordfight]

Anything with high grain should only be encoded in x264 (or just remuxed) as x265 still can not handle it anywhere near as well. I believe the last time someone tested it, they were doing the original Alien, and the re-encode came out larger than the source.

Unless it's magically gotten better since then.
Its gotten alot better then the early days. In my experience x265 have no issue with grain and fine detail, and is imo better then x264 at ”normal” levels (crf18 ~6-9Mpbs if we are discussing 1080p SDR). The issue is that it needs at least preset slow (combined with no-sao and some deblock), and that makes it much slower then x264. I have yet seen x265 be tuned at then same speed as x264 (slower/veryslow with tune film) with better detail retention.

With that said, is x264 better for 1080p sdr to achive near lossless quality at high bitrates? Sure, especialiy if speed also is concidered. But that is not the focus for x265 or most hevc encoders.

[K.i.N.G]

It seams Netflix is working on an AV1 film grain synthesis : AOMedia 2019 Research Symposium - 04 : Netflix (https://youtu.be/M1vwnI0vbMI?t=147).

The danger of that is that many people/companies will use that as an excuse to apply exagerated loss of detail information and 'mask' it with some added synthetic film grain applied to give the false illusion of detail retention (while the information that 'makes sense' is gone). If you know what i mean...
Lets hope they use a good implementation that prevents this as good as possible.

[Fishman0919]

Odd there are no x265 3.2.1 builds around...

[redbtn]

Odd there are no x265 3.2.1 builds around...Yeah, it was released a few days ago but there is no build anywhere.

[agressiv]

Is there any guide to compiling x265 on Windows with an LAVF/Smash patch?

I grabbed the Yuuki source and threw it into my cross-compile script, but get a ton of errors lavf.cpp on and I'm not knowledgeable enough to troubleshoot. I had to hack apart the script as it was to not overwrite the Yuuki version with the current mercurial branch.

One of 50+ errors:
/home/agressiv/ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld: CMakeFiles/cli.dir/objects.a(lavf.cpp.obj):lavf.cpp:(.text+0x11): undefined reference to `avcodec_free_context'

[agressiv]

Yeah, it was released a few days ago but there is no build anywhere.

https://mega.nz/#!vBgRzKbD!SoaO5Lgh9hvtltX4Fgdsp8P3GePNMDVyCiVv9IemkT0

Here's one I compiled - 3.2.1, x64 bit MSVC compiled with 8/10/12 bit support.

[MeteorRain]

Is there any guide to compiling x265 on Windows with an LAVF/Smash patch?

I grabbed the Yuuki source and threw it into my cross-compile script, but get a ton of errors lavf.cpp on and I'm not knowledgeable enough to troubleshoot. I had to hack apart the script as it was to not overwrite the Yuuki version with the current mercurial branch.

One of 50+ errors:
/home/agressiv/ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld: CMakeFiles/cli.dir/objects.a(lavf.cpp.obj):lavf.cpp:(.text+0x11): undefined reference to `avcodec_free_context'

cmake -DENABLE_LSMASH=ON -DENABLE_LAVF=ON -DENABLE_STATIC_LAVF=ON path/

You'll need windows gcc compiled ffmpeg being installed in your x86_64 include & lib.

I have full rakefile to compile Yuuki and Asuna binaries.

[redbtn]

Thank you HolyWu
Can I find somewhere information how to build x265 with gcc or clang? I tried google it, but I fail.

[agressiv]

Yes, thanks HolyWu for the lavf-enabled new build! Any chance you can upload a non-upx packed EXE? Or maybe even your source and compile script?

[fauxreaper]

HME is nondeterministic? Using --hme hex, I got slightly different file sizes on multiple encodes with same input file and x265 config.

[microchip8]

HME is nondeterministic? Using --hme hex, I got slightly different file sizes on multiple encodes with same input file and x265 config.

--hme does not accept any parameters
--hme-search does (where you set the motion estimation algo, such as hex)

[nghiabeo20]

I encode my clip in 5 parts, using -ss and -t to seek to cut position. Now mkvmerge refuses to merge the file, warning that "codec private data doesn't match". After checking the encoding settings, I found that only the --numa-pool change in the first 2 file. How can I edit SPS/PPS data to fool mkvmerge into merging my files? Thanks!

[MeteorRain]

If you don't mind, join files using MP4Box on MP4s.

Other option is to strip the SEI/VPS/SPS/PPS data for part 2-5 and simply join the raw bitstream. (Stripping is optional.)

If you really want to join using mkvmerge, try using a binary editor to change the text inside SEI, however to be honest I don't think that text should make a difference.

[stax76]

Is the estimated file size feature an official feature or a patch? It gets requested frequently, somebody needs to make an official feature request.

[sneaker_ger]

Now mkvmerge refuses to merge the file, warning that "codec private data doesn't match".
I don't think mkvmerge actually refuses to append the files. It is merely displaying a warning message, saying "there may (or may not) be problems".

[foxyshadis]

The danger of that is that many people/companies will use that as an excuse to apply exagerated loss of detail information and 'mask' it with some added synthetic film grain applied to give the false illusion of detail retention (while the information that 'makes sense' is gone). If you know what i mean...
Lets hope they use a good implementation that prevents this as good as possible.

How is that any different from physical film grain on actual film stock? That x264 and x265 completely neglected it because it was hard to test doesn't mean a little noise generation isn't valuable.

[Barough]

x265 v3.2+15-04db2bfee5d6 (https://www.mediafire.com/file/ym8fls6a3dgnfgf/x265-3.2+15-04db2bfee5d6_Win_GCC920.7z/file) (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 9.2.0)
https://bitbucket.org/multicoreware/x265/commits/branch/default

[Forteen88]

x265 v3.2+15-04db2bfee5d6 (https://www.mediafire.com/file/ym8fls6a3dgnfgf/x265-3.2+15-04db2bfee5d6_Win_GCC920.7z/file) (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 9.2.0)Is this version newer than x265-3.2.1+1?

[Barough]

Yes it is, this is compiled from the default branch and not the stable one.

Skickat från min SM-G975F via Tapatalk

[benwaggoner]

How is that any different from physical film grain on actual film stock? That x264 and x265 completely neglected it because it was hard to test doesn't mean a little noise generation isn't valuable.
The grain generation is often done in wrong ways that make it much harder to encode than real world grain. I especially see this in 4K and HDR, where grain gets rendered at the pixel level with SDR code values and so comes out way too bright and too fine-grained. I've seen synthetic film grain that made content impossible to encode at even vaguely acceptable quality at 40 Mbps.

We're also seeing silly amounts of film grain in restored rescans of movies. It isn't creative intent to preserve all kinds of noise that wouldn't have been visible on the perf screens and foot lamberts that the creatives actually approved the content on.

[Blue_MiSfit]

I hear you on the remastering, Ben.

Home Alone is an astounding example of this.

[benwaggoner]

I encode my clip in 5 parts, using -ss and -t to seek to cut position. Now mkvmerge refuses to merge the file, warning that "codec private data doesn't match". After checking the encoding settings, I found that only the --numa-pool change in the first 2 file. How can I edit SPS/PPS data to fool mkvmerge into merging my files? Thanks!
Also maybe try --hrd-concat if you are using HRD?

[Stereodude]

Watch the 4k77 UHD release of Star Wars if you want to see overdone grain. MCTD is calling its name when I build my new Zen 2 system in the next month or two.

[markiemarcus]

I hear you on the remastering, Ben.

Home Alone is an astounding example of this.

Something definitely looked off with Home Alone. The pitch of the grain was both far too fine and too bright; it didn't gel at all with the image.

I must say though, I like the way the UHD release of Predator looks and it's pretty damn grainy. It looks right to me. Was shaky film stock from the outset IIRC.

[RanmaCanada]

The original Predator, in my view, is the PERFECT example of what a transfer and remaster should look like. I'm old enough to have seen it in the cinema, and watching that 4k copy was everything I remembered. And we all know how we view the past with rose coloured glasses.

[LigH]

Film grain ... I witnessed an attempt to preserve a sensible amount of it while trying to remove the majority that only distracts the MPEG-2 encoder of a "Kinowelt" DVD production. Source: Die Feuerzangenbowle (1944) (https://www.imdb.com/title/tt0036818/) – too much denoising, and the trees on the schoolyard look like forged of concrete.

[markiemarcus]

The original Predator, in my view, is the PERFECT example of what a transfer and remaster should look like. I'm old enough to have seen it in the cinema, and watching that 4k copy was everything I remembered. And we all know how we view the past with rose coloured glasses.

Was quite a treat to watch it again I must say. I wonder what the difference in process was, because it obviously wasn't an easy source to work with.

It's a nice encode too. I've noticed that a number of grainy UHD Blu-rays have issues from time-to-time in that regard. There are a handful of scenes in both Blade Runner and The Fog that completely fall apart. Seems inexcusable given how much space there is to play with. In the case of The Fog, the bit rate actually plummets in the affected scenes.