x265 HEVC Encoder

[benwaggoner]

If you're interested in ARM, there is ongoing work for an arm64 port.
You can find all the patches: https://mailman.videolan.org/pipermail/x265-devel/2021-August/013370.html

The full patch is also at the end of that message.

Not sure anyone has compiled them into a build yet..
My fellow Amazonian Sebastian Pop has been doing a whole lot to get ARM performance up! I'll ask him if he has any aggregate numbers for those improvements.

[Barough]

x265 v3.5+12-0983cffc5 (https://www.mediafire.com/file/8sjhyfwgalofcr5/x265-3.5%252B12-0983cffc5_Win_GCC103.7z/file) (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 10.3.0)

https://bitbucket.org/multicoreware/x265_git/commits/branch/master

[LeXXuz]

With all those versions out there and all that talk about things being broken or gotten worse in never versions, I'm really struggling what version I should go for with archival purposes in mind. :rolleyes:

With DJ ATOM's 3.5+20 version (I still don't understand where the +20 comes from anyway) I've noticed very rare and random encoding errors on Ryzen 5950x. Those errors show as a single, (sometimes 2-3 consecutive) frame(s) being almost completely grey.

Worst part is those frames are not really broken and perfectly decodeable which makes them very hard to spot. Usually I let ffmpeg just run through an encoded video stream to spot any decoding errors.
Here I have to watch the entire video closely to check if the encode is flawless or not. Needless to say that's quite a tedious task. :(

[DJATOM]

That +20 is refers to my patches.

[filler56789]

With all those versions out there and all that talk about things being broken or gotten worse in never versions, I'm really struggling what version I should go for with archival purposes in mind. :rolleyes:

My suggestion: get version 3.3+10 from http://msystem.waw.pl/x265/ ,

OR perhaps versions 3.3+1 /+4 /+7 /+12 /+24 /+27 from the x265.exe thread at Videohelp:

https://forum.videohelp.com/threads/357754-[HEVC]-x265-EXE-mingw-builds

[LeXXuz]

That +20 is refers to my patches.

Ah. Thanks for the explanation. :)

My suggestion: get version 3.3+10 from http://msystem.waw.pl/x265/ ,


I'll try my luck with that one. :thanks:

[benwaggoner]

With DJ ATOM's 3.5+20 version (I still don't understand where the +20 comes from anyway) I've noticed very rare and random encoding errors on Ryzen 5950x. Those errors show as a single, (sometimes 2-3 consecutive) frame(s) being almost completely grey.
Is there something specific to your GPU about those errors?

[rwill]

With DJ ATOM's 3.5+20 version (I still don't understand where the +20 comes from anyway) I've noticed very rare and random encoding errors on Ryzen 5950x. Those errors show as a single, (sometimes 2-3 consecutive) frame(s) being almost completely grey.

Worst part is those frames are not really broken and perfectly decodeable which makes them very hard to spot. Usually I let ffmpeg just run through an encoded video stream to spot any decoding errors.
Here I have to watch the entire video closely to check if the encode is flawless or not. Needless to say that's quite a tedious task. :(

Have you tried decoding the stream with a software decoder like the one in ffmpeg ( maybe ffplay ) or the reference software decoder ( ldecode, HM ) to check if its not some decoder side problem ?

[DJATOM]

On my observation rtx 2070's decoder has rare glitches with bframes > 6. Maybe it's also option specific issue...

[benwaggoner]

On my observation rtx 2070's decoder has rare glitches with bframes > 6. Maybe it's also option specific issue...
I've played a whole lot of --bframes 8 content on a Quadro 6000 (basically a 2080 with extra VRAM) without any issues.

[LeXXuz]

Is there something specific to your GPU about those errors?

Have you tried decoding the stream with a software decoder like the one in ffmpeg ( maybe ffplay ) or the reference software decoder ( ldecode, HM ) to check if its not some decoder side problem ?

Hm. Fair point. I forgot to check if this may be decoder related. I already deleted those faulty encodes and encoded those files again with a different version. They look okay now.

If I have time I will try to reproduce that error and check if it's decoder related.

[rwill]

Hm. Fair point. I forgot to check if this may be decoder related. I already deleted those faulty encodes and encoded those files again with a different version. They look okay now.

If I have time I will try to reproduce that error and check if it's decoder related.

It could be a muxer problem too, for example if keyframes are marked as inter and non-keyframes are marked as random access points...

Sometimes I get strange decode errors with VLC too but when I seek to a position prior to the garbage frames and try to reproduce the problems the frames come out ok, sometimes.

[LigH]

Modern decoders should know to seek to IDR frames only, one would assume...

[rwill]

Modern decoders should know to seek to IDR frames only, one would assume...

Well decoders can discard bitstream data until they encounter an IDR but this prevents the case where there are no IDR/CRA/BLA pictures and Intra Refresh is done with constrained Intra encoding like for live communication applications.

The player is relying, in most cases, on what is signaled in the container which contains the bitstream and starts feeding from a point which is marked as a keyframe there.

[LigH]

Thanks for this alternative point of view, @rwill ... video encoding can be more complex than expected when you find a use case where usually optimal techniques don't work perfectly.

[charliebaby]

x265 3.5.0.13 GCC 11.20

LINK : https://www.mediafire.com/file/bzeagt5sbzghwg6/x265-3.5%252B13-4c6880d_gcc112-AVX2.7z/file

[tormento]

x265 3.5.0.13 GCC 11.20
Is AVX2 mandatory? A Sandy Bridge compile would be nice :)

[Barough]

x265 v3.5+15-4bf31dc15 (https://www.mediafire.com/file/xhsb6tmnvndu14g/x265-3.5+15-4bf31dc15_Win_GCC103.7z/file) (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 10.3.0)

https://bitbucket.org/multicoreware/x265_git/commits/branch/master

[LeXXuz]

How much is fixed in 3.5+15 and how much is broken this time? :rolleyes:

[DJATOM]

Latest commits almost did nothing for x86 builds, so not much differs from 3.5 release...

[FranceBB]

Hi there,
I always forget who's maintaining those: http://msystem.waw.pl/x265/

but... can I ask for an AVX-512 Windows x64 build?

[DJATOM]

Afaik all builds supports AVX512, but you have to enable it explicitly (using --avx512 option). And it benefits only in 4k encodes, so it's not really useful for common user.

[FranceBB]

Afaik all builds supports AVX512, but you have to enable it explicitly (using --avx512 option). And it benefits only in 4k encodes, so it's not really useful for common user.

Ah so that's why it's not using them by default, I have to specify it from the command line, got it, now I remember, it's:

--asm=avx512

sometimes I learn something and that pushes out something else I already knew xD

[FranceBB]

New issue: why are 2 pools being created (56 + 56 = 112) but only 1 CPU (56) is used?

https://i.imgur.com/eWyQo1R.png


#Indexing the HDR10 PQ BT2100 4:2:2 10bit planar source
video=FFVideoSource("\\10.64.6.11\avisynth\Scambio\WARNER_MOVIES\BatmanBegins\HDR\BATMANBEGINS_INTL_UHD_HDR_PQ_REC2020_2398_240_PR4444_ITA51_ITA20_EN51_EN20-20210508155521.mov")
ch1=FFAudioSource("\\10.64.6.11\avisynth\Scambio\WARNER_MOVIES\BatmanBegins\HDR\BATMANBEGINS_INTL_UHD_HDR_PQ_REC2020_2398_240_PR4444_ITA51_ITA20_EN51_EN20-20210508155521.mov", track=1)
ch2=FFAudioSource("\\10.64.6.11\avisynth\Scambio\WARNER_MOVIES\BatmanBegins\HDR\BATMANBEGINS_INTL_UHD_HDR_PQ_REC2020_2398_240_PR4444_ITA51_ITA20_EN51_EN20-20210508155521.mov", track=2)
ch3=FFAudioSource("\\10.64.6.11\avisynth\Scambio\WARNER_MOVIES\BatmanBegins\HDR\BATMANBEGINS_INTL_UHD_HDR_PQ_REC2020_2398_240_PR4444_ITA51_ITA20_EN51_EN20-20210508155521.mov", track=3)
ch4=FFAudioSource("\\10.64.6.11\avisynth\Scambio\WARNER_MOVIES\BatmanBegins\HDR\BATMANBEGINS_INTL_UHD_HDR_PQ_REC2020_2398_240_PR4444_ITA51_ITA20_EN51_EN20-20210508155521.mov", track=4)
ch5=FFAudioSource("\\10.64.6.11\avisynth\Scambio\WARNER_MOVIES\BatmanBegins\HDR\BATMANBEGINS_INTL_UHD_HDR_PQ_REC2020_2398_240_PR4444_ITA51_ITA20_EN51_EN20-20210508155521.mov", track=5)
ch6=FFAudioSource("\\10.64.6.11\avisynth\Scambio\WARNER_MOVIES\BatmanBegins\HDR\BATMANBEGINS_INTL_UHD_HDR_PQ_REC2020_2398_240_PR4444_ITA51_ITA20_EN51_EN20-20210508155521.mov", track=6)
audio=MergeChannels(ch1, ch2, ch3, ch4, ch5, ch6)
AudioDub(video, audio)

#From 4:2:2 10bit to 4:2:2 16bit planar
ConvertBits(16)

#From 4:2:2 16bit planar Narrow Range to RGB Planar 16bit Narrow Range
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:2020:2020:limited=>rgb:2020:2020:limited", dither_type="none")

#From PQ to HLG with 16bit precision
Cube("I:\Scambio\WarnerBros\WarnerBros_PQToHLG_MaxCLL_1000.cube", fullrange=true)

#From RGB 16bit planar Narrow Range to YUV420 16bit planar Narrow Range
z_ConvertFormat(pixel_type="YUV420P16", colorspace_op="rgb:2020:2020:limited=>2020:2020:2020:limited", dither_type="none")

#Spatial denoise with 16bit planar precision
dfttest(sigma=64, tbsize=1, lsb_in=false, lsb=false, Y=true, U=true, V=true, opt=0, dither=0)




ffmpeg.exe -i "D:\Temp\AVS Script.avs" -vf scale=out_color_matrix=bt2020nc:out_h_chr_pos=0:out_v_chr_pos=0 -pix_fmt yuv420p16le -strict -1 -an -f yuv4mpegpipe - | x265.exe --y4m - --dither --preset medium --level 5.0 --tune fastdecode --no-high-tier --ref 4 --profile main10 --bitrate 25000 --deblock -1:-1 --min-luma 64 --max-luma 940 --chromaloc 2 --range limited --videoformat component --colorprim bt2020 --transfer arib-std-b67 --colormatrix bt2020nc --overscan show --no-open-gop --min-keyint 1 --keyint 24 --repeat-headers --rd 3 --vbv-maxrate 25000 --vbv-bufsize 25000 --wpp --pass 1 --asm=avx512 -o NUL

ffmpeg.exe -i "D:\Temp\AVS Script.avs" -vf scale=out_color_matrix=bt2020nc:out_h_chr_pos=0:out_v_chr_pos=0 -pix_fmt yuv420p16le -strict -1 -an -f yuv4mpegpipe - | x265.exe --y4m - --dither --preset medium --level 5.0 --tune fastdecode --no-high-tier --ref 4 --profile main10 --bitrate 25000 --deblock -1:-1 --min-luma 64 --max-luma 940 --chromaloc 2 --range limited --videoformat component --colorprim bt2020 --transfer bt2020-10 --colormatrix bt2020nc --overscan show --no-open-gop --min-keyint 1 --keyint 24 --repeat-headers --rd 3 --vbv-maxrate 25000 --vbv-bufsize 25000 --wpp --pass 2 --asm=avx512 -o "D:\Temp\raw_video.hevc"



C:\Program Files\MKVToolNix\mkvmerge.exe --output "D:\Temp\raw_video.mkv" --language 0:und ^"^(^" D:\Temp\raw_video.hevc ^"^)^"

ffmpeg.exe -i "D:\Temp\AVS Script.avs" -map 0:1 -af loudnorm=I=-24:LRA=14:tp=-2 -c:a ac3 -b:a 384k -ar 48000 "D:\Temp\audio.ac3"

ffmpeg.exe -i "D:\Temp\AVS Script.avs" -map 0:1 -af loudnorm=I=-24:LRA=14:tp=-2 -c:a libtwolame -b:a 192k -ar 48000 "D:\Temp\audio.mp2"

ffmpeg.exe -i "D:\Temp\raw_video.mkv" -i "D:\Temp\audio.ac3" -i "D:\Temp\audio.mp2" -map 0:0 -map 1:0 -map 2:0 -c:v copy -c:a copy -c:a copy -f mpegts -metadata provider_name="Sky" -metadata service_provider="Sky" -metadata service_name="Sky" -mpegts_original_network_id 0x1122 -mpegts_transport_stream_id 0x3344 -mpegts_service_id 0x5566 -mpegts_pmt_start_pid 0x1500 -mpegts_start_pid 0x150 -mpegts_service_type "digital_tv" -mpegts_flags "resend_headers" "D:\Temp\final.ts"

pause

[LeXXuz]

Hi there,
I always forget who's maintaining those: http://msystem.waw.pl/x265/


Btw who IS maintaining those?

[FranceBB]

Why am I getting VBV violation? (the command line is the one above). x265 3.5+13-4c6880d

https://i.imgur.com/jSLRrzL.png

[Boulder]

Based on those QPs, your source is just too compressible and the encoder cannot keep up with the requested bitrate at those points?

[FranceBB]

Based on those QPs, your source is just too compressible and the encoder cannot keep up with the requested bitrate at those points?

Makes sense.
The movie is 1h and 30 and it looks like it's near the end, which is pure black with static credits, so it's very likely.
The CBR 1 Gbit/s ProRes is almost definitely zero filled towards that point.
Nobody asked for CBR, so VBR 25 Mbit/s with underflow should be ok. I'm gonna send those over anyway, wish me good luck :P

[Mr.Rippley]

Hi! I decided to learn how to efficiently encode video using HEVC. What is needed for this? Some special software or coding settings should be set "very slowly". Now I use StaxRip 2.8. I would like to achieve the quality and size of the video as on the PSA website, there are just magicians(IMHO). Thank you in advance for your help!

[rwill]

Makes sense.
The movie is 1h and 30 and it looks like it's near the end, which is pure black with static credits, so it's very likely.
The CBR 1 Gbit/s ProRes is almost definitely zero filled towards that point.
Nobody asked for CBR, so VBR 25 Mbit/s with underflow should be ok. I'm gonna send those over anyway, wish me good luck :P

Please check one of the following:
[ ] I know what a VBV underflow means.
[ ] I do not know what a VBV underflow means.

[FranceBB]

Please check one of the following:
[ ] I know what a VBV underflow means.
[X] I do not know what a VBV underflow means.

Checked, but I thought it was implied by the fact that I asked you, guys, about it xD
I get that that any vbv warning means that there's something off with the bitrate, however I generally used to get vbv errors when I was using like a very low CRF and there's a very complex scene like rain or snow etc and the bitrate wanted to skyrocket and fly high but I was limiting it with some constraints to stay much lower than it should, however I feel like this time it's a completely different situation, so if you could clarify what the encoder is actually telling me other than the fact that there's something wrong and it can't abide my constraints, that would be very helpful! :)

[rwill]

Video Buffer Verifier Model: Two buckets with one being above the other and having a hole in it. The buckets
hold bits. The encoder encodes pictures and puts them in the upper bucket. From there they flow through the
hole at a constant rate ( the vbv maxrate ) to the lower bucket. The lower bucket is the video buffer. When the
decoder decodes a picture it takes bits from the lower bucket ( video buffer ), most of the time this happens
at a constant frame rate.

A video buffer overflow happens when a decoder is not taking enough bits from the lower bucket and it flows over.
Now this is not really problematic unless its caused by encoder and decoder clocks drifting apart. If encoder
and decoder share a common clock the encoder should always put pictures at a framerate in its upper bucket
so that they can be removed at the same framerate from the lower bucket. Padding the flow from upper to lower
bucket when the upper bucket is empty may prevent that nothing flows but today thats not really needed anymore,
it was needed in the past though. When replacing the encoder and upper bucket by some instant access storage
( like a file ) one would just stop reading and so stop filling the lower bucket once it is full.

Now a problem arises where the encoder produces pictures that are so large that they cannot make it fast enough
through the hole to the lower bucket and once they made it there they are quicky removed by the decoder at the
fixed rate. There might arise a situation where the decoder tries to fetch a picture from the lower bucket but
the bucket does not contain a complete picture as it still needs some refill from the upper bucket. The past
pictures were to large and got removed previously -> the video buffer is empty, underrun, underflow -> "Buffering".

Video encoders try to prevent such a situation ( mostly at all costs because violating the VBV model will get your
stream get kicked from broadcast multiplex for example ). x265 tries to increase the QP within a picture when it
detects that encoding takes more bits than anticipated while encoding a picutre. I guess the pictures in question
( the ones with the underflows ) show lower quality in the lower pictures parts, maybe even complete breakage.
It appears to be not enough to prevent the underflow though. I think x265 just does not see it coming. Encoding
at such low QPs is quite delicate.

I wonder what the content looks like that x265 is able to maintain an average QP of ~3.2 at that rate. Are you
sure you got your bitdepth and clipping right ? I see in your script that you use
"-pix_fmt yuv420p16le -f yuv4mpegpipe". I never used y4m but wouldnt this flag the y4m as 16 bit depth ?
x265 gets no explicit input bit depth set too. And your .AVS script is doing 10 bit with limited range clipping.
I am confused.

Have you decoded the encoder output ? Does it look ok ?

[FranceBB]

First of all thank you so much for the detailed explanation, it makes a lot of sense now!
Still, I get that x265 just doesn't see it coming, but I kinda thought it would given that it's a two pass encode and this is the second pass.


I wonder what the content looks like that x265 is able to maintain an average QP of ~3.2 at that rate. Are you
sure you got your bitdepth and clipping right ?


Given that it's an old digital movie upscaled by the studio who sent it for the HDR UHD Remaster, I think there are no details whatsoever so it's just very easy to encode for x265.


I see in your script that you use
"-pix_fmt yuv420p16le -f yuv4mpegpipe". I never used y4m but wouldnt this flag the y4m as 16 bit depth ?
x265 gets no explicit input bit depth set too.


Yeah I forgot to post the final AVS, I have updated it now.
And yes, I tend to deliver 16bit planar to x265 and let it dither down to 10bit planar and use x265 itself to perform clipping at 64, 940 Limited TV range (not that HLG ever reaches the upper part of the available 940 anyway 'cause the movie ain't 1000 nits in the first place, but I just play safe). About the complexity, I don't like grain, like, at all, it doesn't look good on anything unless you have a very high bitrate (which I never have), therefore I always always always get rid of it, either spatially or temporally and sometimes both. This also makes things much easier for x265, hence the low qp.


Have you decoded the encoder output ? Does it look ok ?


It looks ok from the RDP given that all this is going on in a server, however I'll be able to tell once the encode is over and the file has been moved to a shared RAID6 instead of a puny local RAID1.

[rwill]

It looks ok from the RDP given that all this is going on in a server, however I'll be able to tell once the encode is over and the file has been moved to a shared RAID6 instead of a puny local RAID1.

I would look at least around timecode 00:34:21, 01:05:02, 01:21:16 and 01:29:24 and check
for visual problems.

Maybe lowering the average rate to 20Mbit or doing more passes will help. Or
increase minQP. x265 does not give many options in such a case, I think its
being too enthusiastic about reaching the average rate of 25Mbit and due to
local failure in frame size prediction it produces underflows.

[Barough]

x265 v3.5+16-7955046e7 (https://www.mediafire.com/file/b0e406dgzwdrcwi/x265-3.5+16-7955046e7_Win_GCC103.7z/file) (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 10.3.0)

https://bitbucket.org/multicoreware/x265_git/commits/branch/master

[Mr.Rippley]

Thanks!

[FranceBB]

Please, help me!

We don't have enough info. Do you want device playback compatibility or not? 'cause if you don't and you just want to compress, then it's much easier. You can try with something like:

--preset placebo --ref 16 --me full --subme 7 --crf 22

to start and then you can work from there.

[Mr.Rippley]

We don't have enough info. Do you want device playback compatibility or not? 'cause if you don't and you just want to compress, then it's much easier. You can try with something like:

--preset placebo --ref 16 --me full --subme 7 --crf 22

to start and then you can work from there.

Thanks for the answer! I tried the placebo preset, coded it for several days) And what kind of encoder program do you use?Or do you consider it the best?

[FranceBB]

Thanks for the answer! I tried the placebo preset, coded it for several days) And what kind of encoder program do you use?Or do you consider it the best?

x265.exe via command line called through a BAT file.
If you're looking for builds, you'll find them here: http://msystem.waw.pl/x265/

[LeXXuz]

'Placebo' preset is a little bit over the top, isn't it?

If encoding time and energy consumption is of no concern, well go for it then.

But I think for private every day use the 'Slower' or 'Very Slow' presets are more than sufficient enough. I highly doubt most people would see any difference, especially on consumer displays.

[RanmaCanada]

Please, help me!

This particular topic is for discussing releases of x265. If you want actual help in learning how to encode with it you could either read the forum itself as there is a tonne of information in it, or create a topic in the forum requesting help while posting as much information as possible.

[Barough]

x265 v3.5+18-747a079f7 (https://www.mediafire.com/file/plqbcfef5j3uml3/x265-3.5+18-747a079f7_Win_GCC103.7z/file) (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 10.3.0)

https://bitbucket.org/multicoreware/x265_git/commits/branch/master

[tuanden0]

Everyone can access to this site http://msystem.waw.pl/x265/?
I got error 404 :(

[RanmaCanada]

Everyone can access to this site http://msystem.waw.pl/x265/?
I got error 404 :(
works here.

[charliebaby]

x265-3.5+19-8003e43_gcc112-AVX2
https://www.mediafire.com/file/zms2i5loo85ca5u/x265-3.5+19-8003e43_gcc112-AVX2.7z/file

[Dogway]

What bitdepth and packing format is used for UHDs? I ask because I plan to do encodes in ICtCp or maybe its variant IPTPQc2. Are there any consumer media in these color models? Do they use 10 or 12-bit, I420, NV12 or something else?

[quietvoid]

What bitdepth and packing format is used for UHDs? I ask because I plan to do encodes in ICtCp or maybe its variant IPTPQc2. Are there any consumer media in these color models? Do they use 10 or 12-bit, I420, NV12 or something else?

UHDs? As in UHD Blu-ray discs?

The only media in IPTPQc2 is premium content on streaming services. It requires access to the Dolby Engine for encoding.
It is a 10 bit video file in the end along with Dolby Vision metadata, and requires playback on a Dolby Vision capable device and display.

There's no consumer media over 10 bit, and it's probably all YUV420.

[Dogway]

I haven't looked into IPTPQc2 prerequisites for encoding, but I know x265 supports ICtCp. With these models 12-bit is encouraged but I'm not sure what kind of hindrances this carries later for compatibility (personal use). As I see all them use Planar 420 (fourcc: I420)

[benwaggoner]

I haven't looked into IPTPQc2 prerequisites for encoding, but I know x265 supports ICtCp. With these models 12-bit is encouraged but I'm not sure what kind of hindrances this carries later for compatibility (personal use). As I see all them use Planar 420 (fourcc: I420)
I don't know of any hardware that can decode 12-bit ICtCp. That's the nominal internal color volume used by DoVi in its intermediate stage, but I've never heard of it being directly encoded nor directly displayed. It's the input for DoVi's tone mapper to get to native panel space.

[Dogway]

As per the papers it's a consumer oriented storage color model since it saves bits (and bandwidth) specially for low luma values and luma/chroma decorrelation is better than YCbCr or YcCbcCrc for HDR.
Yes I don't expect any hardware to be on the edge but I guess if premium stream services include it it must be the streaming apps that decode it to RGB. I personally use the LAVFilters+madVR combo so as far it can be decoded with that I'm fine except madVR's future isn't very bright (not FLOSS).