x265 HEVC Encoder

[ShamisOMally]

@ShamisOMally, burfadel: I redid my comparisons of x265 against x264 and there's no change what comes to x265 smoothing all the detail from certain frames. They probably have not concentrated on that at this point.

You should see how much X265 mangles my Blu-ray of Atlantis: The Last Empire.

The scene with the fireflies with X265 10bit turns into an absolute sea of artifact soup, same with 8bit.

[ShamisOMally]

Isn't PSNR mode designed not for actual practical encoding? How about trying standard settings (non-PSNR) at CRF 18 and see if the same result occurs? Even if this is a clip that contains both static and high motion scenes. With that clip, try also with x265 (non PSNR) at CRF 18.

All you did by disabling PSNR in x265 is to disable adaptive quant, psy-rd, and cutree.

Try default settings in x265, but in addition:
--qg-size 16
--psy-rd 0.8
--weightb
--subme 3
--b-intra
--aq-mode 2
--ref 5
--bframes 6

You could even try a lower CRF, like 17.

For example.

I find enabling PSNR for encodes of my blu-rays I want to stick into my collection absolutely destroys quality due to the encoder emphasizing things incorrectly, like spending more bitrates doing definitions of Text rather than the whole scene.

I actually get more macroblock artifacts with PSNR turned on, most noticeably dark area's like like a sea of dark crap, PSNR turned off the encoder works far better for me

[foxyshadis]

I find enabling PSNR for encodes of my blu-rays I want to stick into my collection absolutely destroys quality due to the encoder emphasizing things incorrectly, like spending more bitrates doing definitions of Text rather than the whole scene.

I actually get more macroblock artifacts with PSNR turned on, most noticeably dark area's like like a sea of dark crap, PSNR turned off the encoder works far better for me

That's normal. PSNR is virtually meaningless for video encoding, it's only useful for micro-optimizations. The tuning only exists to maximize a meaningless number for people who care about that number for historical reasons.

You should see how much X265 mangles my Blu-ray of Atlantis: The Last Empire.

The scene with the fireflies with X265 10bit turns into an absolute sea of artifact soup, same with 8bit.

Are you able to cut out 60s or so of this scene and upload them somewhere? I always appreciate new torture samples. You should be able to use mkvmerge to split just a fragment of the video track.

[ShamisOMally]

That's normal. PSNR is virtually meaningless for video encoding, it's only useful for micro-optimizations. The tuning only exists to maximize a meaningless number for people who care about that number for historical reasons.



Are you able to cut out 60s or so of this scene and upload them somewhere? I always appreciate new torture samples. You should be able to use mkvmerge to split just a fragment of the video track.

Sure, I'll pull out my blu-ray and do a quick redo of the firefly scene

x264 10-bit PSNR turned off
http://i.imgur.com/lIQy69r.jpg

x265 10-bit PSNR turned off
http://i.imgur.com/rMm4s4q.jpg

x265 10-bit PSNR turned on
http://i.imgur.com/OdcOxUB.jpg

[benwaggoner]

The scene with the fireflies with X265 10bit turns into an absolute sea of artifact soup, same with 8bit.
Are you doing preprocessing? If not, I don't recommend changing color depth when reencoding. HEVC doesn't get the same sort of gain as H.264 from encoding in 10-bit.

[benwaggoner]

x265 team set vbv-buff size to twice size the vbv-maxrate. When I set equal size (1x) I saw visible degradation in quality on costly scenes.
That's pretty predictable if the maxrate isn't super high. Presumably you're using >1 sec GOPs. With only a 1 sec buffer, the encoder is limited in its ability to shift bits to a highly detained IDR frame.

I generally prefer to have my bufsize = maxrate * keyint / fps.

Also, maxrate and bufsize are limited by your profile and level in all codecs. For example, Level 4.0 (for 1080p30 and down) has a limit of 12000 Kbps for maxrate and 12000 Kb for bufsize. I'm a strong believer in making compliant streams, so I can yell at someone else if they don't work, so I always specify a level, maxrate, and bufsize in my encodes. They're often high enough to not have a material impact on quality, but I feel better knowing the stream is playable by any compliant decoder.

I still have nightmares from x264 not properly limiting reference frames, and seeing 1080p streams flagged Level 4.0 but with 8 reference frames. The existence of those ~4 years ago is why media apps on Xbox 360 that use Microsoft's official Smooth Streaming component are limited to only 720p.

Which was a bad solution to in my strong opinion. And a good example of why I don't work at Microsoft anymore...

[ShamisOMally]

As you can see with above, when I turn on PSNR it starts defining the edges of lines/objects, which I guess is good? but it washes out the rest of the scene because its allocating bitrate for them.

Problem is it completely loses all detail elsewhere, the number of fireflies that vanish when PSNR is turned on is rather drastic, look around the two men's hands and the far right side, PSNR turned on a great deal of them vanish into artifacts

[ShamisOMally]

That's pretty predictable if the maxrate isn't super high. Presumably you're using >1 sec GOPs. With only a 1 sec buffer, the encoder is limited in its ability to shift bits to a highly detained IDR frame.

I generally prefer to have my bufsize = maxrate * keyint / fps.

Also, maxrate and bufsize are limited by your profile and level in all codecs. For example, Level 4.0 (for 1080p30 and down) has a limit of 12000 Kbps for maxrate and 12000 Kb for bufsize. I'm a strong believer in making compliant streams, so I can yell at someone else if they don't work, so I always specify a level, maxrate, and bufsize in my encodes. They're often high enough to not have a material impact on quality, but I feel better knowing the stream is playable by any compliant decoder.

I still have nightmares from x264 not properly limiting reference frames, and seeing 1080p streams flagged Level 4.0 but with 8 reference frames. The existence of those ~4 years ago is why media apps on Xbox 360 that use Microsoft's official Smooth Streaming component are limited to only 720p.

Which was a bad solution to in my strong opinion. And a good example of why I don't work at Microsoft anymore...

I have buffer at absolute min because I do in home viewing and not web streaming, I'll encode the scene again with buffer set to max to see if it changes anything with x265 10bit

*EDIT* If I had any real input on the development on codecs, I would ask why x264/x265 is absolutely TERRIBLE with blue <--> purple transitions and why its not been fixed/adjusted for it.

I mean, wow, every encode I've ever done, its the first place render artifacts come up.

[nandaku2]

As you can see with above, when I turn on PSNR it starts defining the edges of lines/objects, which I guess is good? but it washes out the rest of the scene because its allocating bitrate for them.

Problem is it completely loses all detail elsewhere, the number of fireflies that vanish when PSNR is turned on is rather drastic, look around the two men's hands and the far right side, PSNR turned on a great deal of them vanish into artifacts

Tune psnr disables AQ, psy-rd and psy-rdoq. This is to be used only if you're interested in objective PSNR measurements, as PSNR hates it when bits are redistributed across a frame.

Since you care about subjective visual quality, please try the default settings.

[foxyshadis]

I mean, can you upload the actual original blu-ray version of the firefly scene? Otherwise I can rent the disc and find it, it's not that big of a deal. There's something really, really wrong with the way x265 treats that scene.

[ShamisOMally]

x265 10-bit PSNR off 16000Kbit buffer
http://i.imgur.com/oJBh8x0.jpg

Massive quality increase, but image is still not as good as x264 which was only using a buffer of 64kbit buffer

@Foxy: I'd rather not upload clips, as I'd likely get in shit from some law crap or what not for doing so

[benwaggoner]

x265 10-bit PSNR off 16000Kbit buffer
http://i.imgur.com/oJBh8x0.jpg

Massive quality increase, but image is still not as good as x264 which was only using a buffer of 64kbit buffer

@Foxy: I'd rather not upload clips, as I'd likely get in shit from some law crap or what not for doing so



Can you share your complete command lines for x264 and x265? Note that there isn't a --tune animation for x265 yet, although there are recommended anime settings floating around.

[ShamisOMally]

Can you share your complete command lines for x264 and x265? Note that there isn't a --tune animation for x265 yet, although there are recommended anime settings floating around.

x265 settings:

x265_10bpp --preset veryslow --profile main10 --level-idc 50 --tune psnr --pmode --pme --no-open-gop --keyint 250 --scenecut 40 --wpp --lft --sao --vbv-maxrate 70000 --vbv-bufsize 16000 --crf 18 - "$(DestFile)"

x264 settings:

x264_10bpp --no-progress --profile high10 --preset veryslow --tune psnr --keyint 500 --min-keyint 1 --no-dct-decimate --no-deblock --non-deterministic --no-psy --vbv-maxrate 70000 --vbv-bufsize 64 --crf 18 --sar 1:1 --threads 18 -o "$(DestFile)" -

*EDIT* If you guys want me to make a sample recording of a game etc I can do that and I'll throw whatever settings you want at it, just built a new encoding server for lets plays.

*EDIT2* I never tune for animation, I find it ruins small details and just enhances edges

[sneaker_ger]

Same crf comparisons are useless, you need to do same bitrate. Watch for x264 bufsize error, 64 is too low to fit one frame so x264 will increase automatically. Watch for x265 maxrate error. (You've already been told not to use --tune-psnr)

--lft is not in doc, is it deprecated?

[ShamisOMally]

--lft is not in doc, is it deprecated?

*shrugs* I'm just trying to solve a mystery, I only really know x264 settings, not x265.

Also you've lost me on the whole "CRF is not a good indication of performance" because I thought x265's strength was it was supposed to be near half bitrate for same quality. And I was getting that, x265 CRF 18 was outputting at half the file size of x264. Well, until I cranked vbuffer up to insanity levels, but it still didn't get the same quality.

*EDIT* BBL after a sleep, taking parents to doctors tomorrow

[sneaker_ger]

CRF is not an absolute metric for quality. You cannot set x264 --crf 18 and x265 --crf and expect identical quality. You even confirmed it yourself: qualities varied even though you always used --crf 18.

I thought x265's strength was it was supposed to be near half bitrate for same quality.
Irrelevant (and often wrong)

[LigH]

--lft is not in doc, is it deprecated?

It is.

x265cli.h
{ "no-lft", no_argument, NULL, 0 }, /* DEPRECATED */
{ "lft", no_argument, NULL, 0 }, /* DEPRECATED */

param.cpp
OPT("lft") p->bEnableLoopFilter = atobool(value); /* DEPRECATED */

@ ShamisOMally: Do yourself a favour to use only parameters you understand; especially only if you are really sure that they must override preset defaults.
__

A first release with "multilib" EXE: x265 1.7+234-b1af4c36f48a (https://www.mediafire.com/download/m4464m18pk44o8b/x265_1.7+234-b1af4c36f48a.7z)

[ShamisOMally]

@ ShamisOMally: Do yourself a favour to use only parameters you understand; especially only if you are really sure that they must override preset defaults.


I use Mediacoder as the front end, it was something on I didn't know was on.

*EDIT1* Also more new to x265 than I am to VP9, which I've had months playing with perfecting its use/quality

[stax76]

A first release with "multilib" EXE: x265 1.7+234-b1af4c36f48a (https://www.mediafire.com/download/m4464m18pk44o8b/x265_1.7+234-b1af4c36f48a.7z)

This is very helpful because it removes the need of picking the right files and renaming the exe, also the compressed size is slightly smaller.

[LigH]

I'll still include DLLs in case anyone prefers to link them in an application... theoretically... does anyone?

I guess 7-zip will compress them all together good enough. If you want to ship only one EXE, decide for yourself whether or not to UPX it. And yes, UPX will probably have an easy job compressing two very similar libraries in one EXE.

[Atak_Snajpera]

Thanks for testing. It is time to dig deeper...

Can you at least compile whole set (x86/x64/8bit/10bit) of binaries with your patch?

[Barough]

A first release with "multilib" EXE

Thnx for the "multilib" EXE's. :)

[Ma]

Can you at least compile whole set (x86/x64/8bit/10bit) of binaries with your patch?

I did so in version 1.7+234
www.msystem.waw.pl/x265/

But I made only one EXE for 8 & 10-bit encoding -- multilib.

[Atak_Snajpera]

I did so in version 1.7+234
www.msystem.waw.pl/x265/

But I made only one EXE for 8 & 10-bit encoding -- multilib.

Is Main10 also supported?

http://i.cubeupload.com/Zzt6SD.png

[Ma]

Is Main10 also supported?

Yes, please add -D 10 option.

[LigH]

Yes, with the output depth parameter (-D 8|10).

I just ran a test with a high-res video (Tears of Steel, 60 seconds action) with almost the parameters of ShamisOMally:

--preset slow -D 10 --crf 18 --vbv-maxrate 70000 --vbv-bufsize 16000

To be honest ... I cannot imagine ***ing up parameters so badly that x265 would produce such a mess at CRF 18. Whatever these screenshots belong to, I doubt they belong to the documented parameter sets. My result of ToS was "visually transparent".

I wish I had a good cartoon scene to check it with. But still, I don't know any copyleft cartoon source with little quality loss.

[x265_Project]

x265 settings:

x265_10bpp --preset veryslow --profile main10 --level-idc 50 --tune psnr --pmode --pme --no-open-gop --keyint 250 --scenecut 40 --wpp --lft --sao --vbv-maxrate 70000 --vbv-bufsize 16000 --crf 18 - "$(DestFile)"

x264 settings:

x264_10bpp --no-progress --profile high10 --preset veryslow --tune psnr --keyint 500 --min-keyint 1 --no-dct-decimate --no-deblock --non-deterministic --no-psy --vbv-maxrate 70000 --vbv-bufsize 64 --crf 18 --sar 1:1 --threads 18 -o "$(DestFile)" -

*EDIT* If you guys want me to make a sample recording of a game etc I can do that and I'll throw whatever settings you want at it, just built a new encoding server for lets plays.

*EDIT2* I never tune for animation, I find it ruins small details and just enhances edges

The main problem: --vbv-maxrate 70000 --vbv-bufsize 16000

The artifacts you're seeing are related to rate control, specifically, VBV raising QP very high to limit the bit rate in order to comply with your VBV parameters. You are setting vbv-bufsize too low in comparison to vbv maxrate, and to a CRF 18 encode. The VBV algorithm has no room to work, and it's going to crush your video in complex scenes. VBV is meant to be used to limit peak bit rates in order to enable hardware decoders to function properly. If you're decoding your H.265 content on a PC (in software), you probably don't need to use VBV to limit bit rate peaks. If you really want to use VBV for your encodes, set VBV maxrate to some reasonable number (at least 1.5x, but more likely 2x your average bit rate), and VBV bufsize to a value that is greater than VBV maxrate (2x the maxrate is a good value).

When you use VBV maxrate and bufsize values that are out of line with your other settings, you will get bad results. If you want to see what's happening, turn on frame level logging (--csv logfilename.csv --csv-log-level 2), and you will see the QP values and frame sizes. Do this first for a CRF encode without VBV, then repeat the process with VBV, and you'll see the effect that VBV has on your encodes, frame by frame. You can plot the QP values on a spreadsheet to see when VBV takes effect, and you can plot the frame sizes or a moving average bit rate to see the effect on bit rate in complex scenes.

As others have mentioned, you should never use --tune psnr for any actual encodes that you intend to watch. This should only be used for PSNR measurements when you want to compare x265 PSNR measurements to another encoder.

[benwaggoner]

Yeah, strip everything out of those command lines other than preset and no-open-gop. They'll look a lot better and encode faster, particularly x265. Even on an 18-core/36-thread system pmode can slow things down and pme definitely will. The rest of the stuff in both either makes for invalid comparisons, hurts quality, or is already on by default at very slow.



Since you are running at 1080p, Level 4.0 is probably appropriate, and ample for perfect anime reproduction.



Comparing at the same bit rate is a good start. Using CBR (bitrate=maxrate, bufsize set 12000 in x265 and 31250 in x264) would be a good start since you can see local variance more easily. X265 does have the advantage of a bigger bufsize by the design. Maybe try 1000, 2000, 4000, and 8000?



Also, since you have an 8-bit source, I'd start with encoding 8-bit directly from the source decor (yuv420p in and out), which is easier to preview and gets preprocessing out of the equation.



There is additional quality tuning that can be done for each, but the above would be a good baseline without any content-specific tweaks.

[ShamisOMally]

Yes, with the output depth parameter (-D 8|10).

I just ran a test with a high-res video (Tears of Steel, 60 seconds action) with almost the parameters of ShamisOMally:

--preset slow -D 10 --crf 18 --vbv-maxrate 70000 --vbv-bufsize 16000

To be honest ... I cannot imagine ***ing up parameters so badly that x265 would produce such a mess at CRF 18. Whatever these screenshots belong to, I doubt they belong to the documented parameter sets. My result of ToS was "visually transparent".

I wish I had a good cartoon scene to check it with. But still, I don't know any copyleft cartoon source with little quality loss.

Actually, MOST of the movie was amazingly watchable, but that firefly scene, god damn it just turned into soup

*EDIT* If you guys are looking for something comparable to test with the same degree of "AHHH" in about 2 hours I'll do some high end 1080p 60FPS raw video samples of L4D2 Dark Carnival Stage 5, the fireworks going off on stage is probably the most brutal thing I've ever had to encode for youtube/webm's etc without raising the bitrate to insane levels to compensate, tons of white/yellow/red firework sparks in a dark stadium

*EDIT2* Sorry, going to be at least 3-4 hours. Encoding some lets play footage

[Ma]

I will surprise you ;) None of them work.

Good news is that I'm able to reproduce this bug on my 4-threads CPU. Normally x265 uses on my CPU only 2 frame threads, on your CPU -- 5. If I add -F 5 option, the stat file is corrupted.

Edit: After deep digging: the x265 source code is OK, GCC is OK, the bug is in MinGW-w64 fprintf function implementation. We can make workaround for this bug (restrict threads or use atomic function like fputs for example). I informed MinGW-w64 team about this issue
https://sourceforge.net/p/mingw-w64/discussion/723797/thread/55520785/

[Selur]

Seeing that '--output-depth' over at https://x265.readthedocs.org/en/latest/cli.html?highlight=#cmdoption--output-depth only states 8 and 10 I was wondering if 12 and 16 bit encoding was abandoned.

[LigH]

Why abandoned? Rather not yet started; but why document an unimplemented feature? Someone could try to use it and complain that it does not yet exist, even though it was mentioned in the documentation... so better not even mention it as long as its implementation hasn't even started yet.

[Selur]

12bit and 16bit encoding did work last time I tested it, only the asm optimizations weren't up-to-date,...

[Ma]

x265 sometimes creates corrupted stat file

Finally I've made ultimate fix of this bug -- without any changes in source code. I turn off MINGW_ANSI_STDIO and link to msvcr120.dll (without MinGW extension use of msvcrt.dll is danger). It is working without any patches! So there is no bug.

www.msystem.waw.pl/x265/test3-Atak.7z

[LigH]

That means you created a new dependency to the Microsoft Visual C++ Runtime in a specific version, which people would have to install, even though the executable was not built by Visual Studio?

[stax76]

Atak could verify the VC++ 2013 runtime dependency in his installer like he does for Java and ffdshow, I do something similar. The original AviSynth+ installer includes both VC++ 2013 x64 and x86, I removed them both from my customized AviSynth+ installer and verify VC++ like all external tools and dependencies. With tweaks like this I brought my download size from 90 MB down to below 60 MB. There is a high chance that VC++ 2013 is already installed as it's currently the most popular version and if it's missing a app can easily guide the user to install it assuming the app can start without it, in case something is missing StaxRip shows it's apps dialog asking to install it:

http://thumbnails108.imagebam.com/41765/f9b9bb417643505.jpg (http://www.imagebam.com/image/f9b9bb417643505)

[Ma]

That means you created a new dependency to the Microsoft Visual C++ Runtime in a specific version, which people would have to install, even though the executable was not built by Visual Studio?

Yes, I replaced msvcrt.dll dependency to msvcr120.dll (they both are MSVC runtimes). msvcrt.dll start with Win95 OSR2 and it not even support C99 standard. It is a source of the problem.

I don't know which path is better: fix mingw-w64 and stay with msvcrt.dll or switch to msvcr100/110/120.dll.

[LigH]

Or third: Implement a custom stats file output routine to circumvent it... shooting sparrows with cannons?

[foxyshadis]

12bit and 16bit encoding did work last time I tested it, only the asm optimizations weren't up-to-date,...

Check again, >10bit doesn't exist in x265 yet, barring a couple of experimental builds for rExt. Are you sure you're not thinking of --input-depth, which dithers down to 10bit?

[Atak_Snajpera]

Ma Your earlier patch works fine so no need to mess with compiler options. x265 team should implement your workaround because without it gcc builds are semi broken. However it is iteresting why non gcc builds work fine...

[LigH]

@ Selur + foxy:

That's why I halted my reply so far. I knew there are routines for Main (8 bit) and Main10 (10 bit) HEVC profiles. I am not sure if there are finer resolutions for internal encoding parameters mentioned in HEVC specs, I only heard about it in theory so far but couldn't remember any specific implementation (of course not in assembler, but not even in C), and I also wondered if this would be even another HEVC profile if "Main10" is already so specific in its name.

@ Ma + Atak:

It would be a pity if non-Microsoft builds would require Microsoft DLLs. And this is only a workaround for Windows; what would you have to substitute if there were a similar issue only with GCC builds on Linux, assuming there are also other Linux compilers? I would hope for a mostly compiler independent solution which will certainly require understanding of the reason. But I fear it may require severe efforts and background knowledge in thread-safe development. A reply from the team will be welcome.

[sneaker_ger]

I also wondered if this would be even another HEVC profile if "Main10" is already so specific in its name.

Indeed there are lots of specific profiles.
https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding#Profiles

[LigH]

So we can conclude: Only profiles "Main" and "Main 10" are implemented (to some extent) in x265 so far; implementation of other profiles (like "Main 12") is possible (and probably anticipated during the development of currently selected profiles), but not yet available.

Correct so far?

[x265_Project]

12 bit support is on our near-term roadmap.

I'm curious - do any of you have any real need for 12 bit HEVC (and do you have source video content that has at least 12 bits/color sample precision)? Or are you just interested in trying it?

[LigH]

Hey, there are also completely confident users of "placebo" presets. If I ever want to top that, I'll call my custom preset "homeopathic" (= "placebo forte"). :rolleyes:

No, getting real, I would not recommend to spend too much time on 12 bit precision now, as long as people still mourn about e.g. detail loss in hopelessly low bitrates.

[x265_Project]

Hey, there are also completely confident users of "placebo" presets. If I ever want to top that, I'll call my custom preset "homeopathic" (= "placebo forte"). :rolleyes:

No, getting real, I would not recommend to spend too much time on 12 bit precision now, as long as people still mourn about e.g. detail loss in hopelessly low bitrates.

I hear you, but we have paying customers who insist that we focus on certain things in our development roadmap, and one of these things is support for 12 bit precision. This isn't for consumer video applications. It's needed for storing or delivering pro video in the highest possible quality.

For consumers, HEVC with more than 10 bits of precision might be applicable for still image compression.

[kolak]

12 bit support is on our near-term roadmap.

I'm curious - do any of you have any real need for 12 bit HEVC (and do you have source video content that has at least 12 bits/color sample precision)? Or are you just interested in trying it?

You can find footage which was shot/processed and final delivered at 12bit. It will be mainly some film content, but even this quite often ends up just 10bit. Current film pipes are based on 32bit float EXR format, so if source comes from very good camera than there may be real 12bit info in final master, but someone could question it. High-end cameras store raw data at 10-16bit, but it's not that obvious that data there has real 12bit. Viewing it is yet another story.

[MeteorRain]

I'm curious - do any of you have any real need for 12 bit HEVC (and do you have source video content that has at least 12 bits/color sample precision)? Or are you just interested in trying it?

Interested in how much more efficiency 12 bit can bring us, especially on animations.

[foxyshadis]

12- and 16-bit are more for video grading, medical imaging, that sort of application, where you want the flexibility to wildly change the dynamic range between the captured and the final version. In video, ProRes all but owns this space right now, with its 12-bit 444+4 codec, like Cineform and DNxHD before it, so there's still a very substantial market to crack into. For deep images, JPEG2000 is still the dominant format.

For fully graded end-user video, the size or quality difference between 10- and 12-bit would be vanishingly small. Your monitor is the limiting factor.

[benwaggoner]

Interested in how much more efficiency 12 bit can bring us, especially on animations.
I doubt any 8-bit was at about the perceptual threshold, and sometimes not enough. For Rec. 709, 10-bit is ample.

Also, very little cel animation has been created at >10-bit anyway. Maybe a few films from the last decade, and recent remasters of old film sources.

Maybe if there is HDR animation some day, a 12-bit PQ curve would be better than 10-bit PQ. But I've never heard even a rumor of anyone even contemplating high dynamic range cel animation.