x265 HEVC Encoder

[littlepox]

Originally Posted by benwaggoner

Has anyone tried doing a separate "tune animation." Based on x264, I'd think optimal cel animation settings would be substantially different than the film ones. Although there may be some overlap in constraining unit size and such.

We have that as well. For anime contents basically you wish to encourage bits to be invested in the edge areas, and that can be done by tweaking psy and aq.


Will probably do another extensive test and post our suggestions here after v2.0. Our current parameters are updated with v1.9 stable.

[pingfr]

@littlepox: No feedback or conclusions on the last encode that ended up a few hours ago?

[littlepox]

Originally Posted by pingfr

@littlepox: No feedback or conclusions on the last encode that ended up a few hours ago?

No. This was just a repeat of our previous tests suggesting higher presets shall only waste your time with little improvement.

[pingfr]

@littlepox: And it turned out we both came to the same conclusions, more or less.

I'm currently pondering whether should I request more funds from our IT department boss to upgrade to a dual Xeon E5-26xx v3 right away or if I should instead wait a few extra months/weeks for Intel to "release" their upcoming and highly anticipated newer Broadwell-EP v4 lines of Xeon CPUs.

It would help me decide if I knew what kind of fps crunching improvement I would get... but meh, I disgress.

Thanks for all the fish!

[Boulder]

Out of interest, do you have to keep everything at the original resolution? Upon playback, a huge amount of Blu-rays look just fine, if not the same as the original, when downscaled to 720p with some sharpening.

Originally Posted by pingfr

I'm currently pondering whether should I request more funds from our IT department boss to upgrade to a dual Xeon E5-26xx v3 right away or if I should instead wait a few extra months/weeks for Intel to "release" their upcoming and highly anticipated newer Broadwell-EP v4 lines of Xeon CPUs.

v4 Xeons (Broadwell) use the same basic CPU architecture as v3 (Haswell), so you shouldn't expect to see much in the way of instructions-per-clock improvement. You will be able to get up to 22 cores per chip with v4 Xeons, but it's likely that the price per core will probably be similar for a while after the v4 release. v5 Xeons (Skylake) will support AVX-512 and higher internal ring bandwidth, but I don't expect to see them until late 2016 or early 2017.

[pingfr]

@x265_Project: Hey Tom, good to have you here.

So basically what you're saying is, that a dual v3 encoding box will more or less yield identical encoding speed (fps crunching) to a dual v4 Broadwell-EP because there aren't any newer set of instruction introduced?

Originally Posted by pingfr

@x265_Project: Hey Tom, good to have you here.

So basically what you're saying is, that a dual v3 encoding box will more or less yield identical encoding speed (fps crunching) to a dual v4 Broadwell-EP because there aren't any newer set of instruction introduced?

Hey... I'm always here... but I'm letting our experts, nandaku2 and pradeeprama handle more of the technical questions.

That's right... Broadwell is basically a die shrink of Haswell, going from 22 nm to 14 nm. As with any new chip, there will be refinements in timings and power utilization... but the logic units and architecture are basically the same, and there are no new instructions in Broadwell. Because of the die shrink, however, (according to leaked information) there will be larger Xeons available with more cores.

On the consumer side there were more changes... primarily bigger, better GPUs (Intel Gen Graphics).

[LigH]

Originally Posted by djesteban

I am having trouble with this too... and trouble finding answers.
I was able to make zones work in CRF mode, but it doesn't seem to have any effect on 2pass encodes.

Can anyone confirm this?!

I just tested the Sintel trailer in a smaller size (640x272) in 3 passes with verbose CSV log files. Plotting the average QP per frame (in encoded order only, displayed order is too hard to reconstruct in Excel) reveals a quite confusing behaviour in "--pass 2", while it looked about as expected in "--pass 3". Developers got details via mailing list.

[aegisofrime]

Originally Posted by x265_Project

Hey... I'm always here... but I'm letting our experts, nandaku2 and pradeeprama handle more of the technical questions.

That's right... Broadwell is basically a die shrink of Haswell, going from 22 nm to 14 nm. As with any new chip, there will be refinements in timings and power utilization... but the logic units and architecture are basically the same, and there are no new instructions in Broadwell. Because of the die shrink, however, (according to leaked information) there will be larger Xeons available with more cores.

On the consumer side there were more changes... primarily bigger, better GPUs (Intel Gen Graphics).

While we are on this topic, what would you say would be the performance increase upgrading from a i7-4770 to a i7-6700?

Originally Posted by aegisofrime

While we are on this topic, what would you say would be the performance increase upgrading from a i7-4770 to a i7-6700?

For most scenarios, the Skylake i7-6700 is about 20% faster, due to the higher clock speed. For 4K ultrafast it is more than 2x faster, due to higher internal memory bandwidth (which bottlenecks the Haswell i7-4770).

[pingfr]

Since we're talking hardware, I'll drop my experiments here:

In the lab we're using a 5960X (8C/16T) which has it's stock speed clocked at 3GHz with a Turbo speed of 3.5GHz, I instantly overclocked it to 4GHz with a very basic cooler and without any extra tweaks (I could most likely push it farther towards the 4.5GHz even 4.7GHz clamp with proper watercooling, which in my case can't be justified in a professionnal environment).

During our crunching tests at the 4GHz frequency, I use a reference short 5 minutes (11862 frames@50fps) Ultra HD clip with the nasty resolution of 3840x2160; with the following parameters on x265-1.9+88 (10 bits):

avs4x265.exe -P x265-10b.exe -D 10 --input-depth 10 --fps 50 --input-res 3840x2160 --preset ultrafast --no-rect --crf 18 --output output.hevc %1

We're getting a 8.80 fps crunching speed, rendering the 11862 frames encoded more or less under 22 minutes.

This sunday I was granted temporary access to a "supposedly" high-end machine running a single Xeon E5 2695v3 at stock 2.2GHz (could not be overclocked at all, wasn't allowed to play around with it's BIOS or anything) and let's just say that I was REALLY REALLY disappointed.

During my tests, the Xeon E5 2695v3 running at full speed (2.2GHz) with all cores saturated at 100% at all time, gave us crunching speeds of 4.5fps which seemed abysmally low for a production server that was far more costly than our little lab's i7 5960X.

I gathered that a CPU running stable at 4GHz will pretty much "own" a 2.2GHz clocked CPU, but at the same time, that Xeon is supposed to sport 14 cores and no less than 28 threads which I initially thought would grant faster speeds over an i7 HEDT sporting "only" 8 cores and "only" 16 threads and therefore "make up" for the slower clocking speeds.

Could it be related to the fact we only found out in the end that Xeon CPU was an Engineering Sample leaked from Intel and therefore could be slower/crippled/lacking features over a retail CPU?

Who knows.

As far as I'm concerned: i7 5960X (8C/16T) OC'ed at 4GHz > Xeon E5-2695v3 ((ES) 14C/28T) at 2.2GHz any time, any day, anywhere.

[djesteban]

Originally Posted by LigH

I just tested the Sintel trailer in a smaller size (640x272) in 3 passes with verbose CSV log files. Plotting the average QP per frame (in encoded order only, displayed order is too hard to reconstruct in Excel) reveals a quite confusing behaviour in "--pass 2", while it looked about as expected in "--pass 3". Developers got details via mailing list.

LigH, I have a thread dedicated to this problem here.
Would be nice if you could post your result there also to attract more attention on the issue.

I'll make my own test on my side and post my result there. If there's indeed an issue, I'll log a bug on bitbucket I guess...

Originally Posted by pingfr

In the lab we're using a 5960X (8C/16T) which has it's stock speed clocked at 3GHz with a Turbo speed of 3.5GHz, I instantly overclocked it to 4GHz with a very basic cooler and without any extra tweaks...

During our crunching tests at the 4GHz frequency, I use a reference short 5 minutes (11862 frames@50fps) Ultra HD clip with the nasty resolution of 3840x2160; with the following parameters on x265-1.9+88 (10 bits):

avs4x265.exe -P x265-10b.exe -D 10 --input-depth 10 --fps 50 --input-res 3840x2160 --preset ultrafast --no-rect --crf 18 --output output.hevc %1

We're getting a 8.80 fps crunching speed, rendering the 11862 frames encoded more or less under 22 minutes.

This sunday I was granted temporary access to a "supposedly" high-end machine running a single Xeon E5 2695v3 at stock 2.2GHz (could not be overclocked at all, wasn't allowed to play around with it's BIOS or anything) and let's just say that I was REALLY REALLY disappointed.

During my tests, the Xeon E5 2695v3 running at full speed (2.2GHz) with all cores saturated at 100% at all time, gave us crunching speeds of 4.5fps which seemed abysmally low for a production server that was far more costly than our little lab's i7 5960X.

I gathered that a CPU running stable at 4GHz will pretty much "own" a 2.2GHz clocked CPU, but at the same time, that Xeon is supposed to sport 14 cores and no less than 28 threads which I initially thought would grant faster speeds over an i7 HEDT sporting "only" 8 cores and "only" 16 threads and therefore "make up" for the slower clocking speeds.

As far as I'm concerned: i7 5960X (8C/16T) OC'ed at 4GHz > Xeon E5-2695v3 ((ES) 14C/28T) at 2.2GHz any time, any day, anywhere.

You're seeing Amdahl's Law in action. While we've done everything possible to parallelize x265, there are certain functions that are inherently serial (CABAC encoding and decoding, for example). Even for the operations we've parallelized, there are often dependencies on other threads.

For example, if you are running 8 threads on a quad-core machine, most threads will be busy most of the time. Occasionally, one or more of the 8 threads will be waiting for the result from another thread, and so it will be stalled. If you double the power of your machine (run a 5960x with 8 cores/16 threads), you will not get 2x the performance, due to Amdahl's Law. Now, the number of threads that might be waiting for another thread to finish goes up. In x265 we have multiple frame encoders, which call row encoders, which call CTU encoders. Let's say one of the CTU encoders encounters some complex video, and it doesn't easily find good prediction matches in its reference frames. It will take longer than average to encode, and it will stall the row encoder, which may stall other row encoders that depend on that CTU being done (so that it can be referenced).

So, it is an ongoing challenge to get the highest possible effective utilization from many-core machines. We have a private commercial encoding library called UHDkit that can break the incoming video into chunks, encoding each chunk with a separate instance of x265. In this way we can gain higher effective utilization on many-core machines, such as the dual Xeon E5-2699 v3 (2 x 18 cores = 36 cores / 72 threads). The downside is added latency for live encoding, as you have to queue up enough video to keep all encoder instances busy.

[pingfr]

Originally Posted by x265_Project

We have a private commercial encoding library called UHDkit that can break the incoming video into chunks, encoding each chunk with a separate instance of x265.

Hey Tom,

Sounds good but on our little test-bed project, I already fought hard with my department's manager and the IT department's boss/office chief to allocate funds to acquire a 5960X...

In the coming days I will have to fight again to convince them to either allocate even more extra funds or to grab a dual E5-26XX-v3 server from the next IT department...

So now if on top of that, I also have to convince them to acquire expensive licenses for a private commercial encoding library... my head will be on a pike, literally.

But yes, this definitely looks sexy:



:drool:

[stax76]

StaxRip should now be up-to-date in regard of x265, changes in the last build are:

• larger custom command line TextBox
• new switch added --rc-grain
• tune grain defaults updated
• the encoding options dialogs for x265, AMD, Intel and NVIDIA have now an option to display the full command line
• removed one switch that was accidentally added twice :-)
• x265 1.9+88

Download: http://forum.doom9.org/showthread.ph...40#post1760640

[pingfr]

@stax76: Congrats! (I'm more of a MeGUI guy myself but nevertheless, grats to you!).

Edit: Also good to see those settings littlepox gave us a few days ago are being added as a "custom" profile to StaxRip, way to go!

[benwaggoner]

Originally Posted by littlepox

Will probably do another extensive test and post our suggestions here after v2.0. Our current parameters are updated with v1.9 stable.

I don't know there's any indication that 1.9 will be followed by a "super major" 2.0 release, other than base 10 . Next up might be 1.10 AFAIK. Or maybe 1.A if it turns out MCW was using hex all along...

1.9 is a good release, and I don't think there's any reason to wait to make animation tuning. And the earlier it is worked on, the more feedback can go into the next version. I'd love to see a proper --tune animation and --tune film in there .

[LigH]

@ benwaggoner:

The x265 team already documented that they will prefer a continuous decimal version progression. After v1.9 will quite certainly follow v2.0 without any "super major" speciality attribute, v1.0 wasn't special either after v0.9.

[pingfr]

@LigH: How about... x265-v2.0-reloaded-extended-unleashed-extreme-edition-super-combo-deluxe+1 ?