HEVC multi-rate encoding for adaptive HTTP streaming

[damjeux]

I haven't seen much talk about multi-rate encoding with HEVC. Basically, a single system which encodes one video at different bitrates to target adaptive bitrate streaming. The different bitrates can be achieved by encoding at different spatial resolutions, different signal qualities or different frame rates. Sharing information between the individual encoder processes can lead to an overall computational complexity reduction.

I have recently implemented a multi-rate encoding method for x265 (https://github.com/damjeux/multi-rate-x265/commit/4063be252750d93a06992c898dd8133ad8ebf94c) based on a CU structure reuse from a high-quality encoding to speed-up lower quality encodings.
I have also posted some results in a blog post (https://damienschroeder.wordpress.com/2016/08/10/multi-rate-hevc-with-x265-for-adaptive-http-streaming/). Depending on the configuration, the x265 multi-rate encoder can for example achieve an overall complexity reduction of 10% with no rate-distortion loss (even a very small improvement on average). Or achieve 20% complexity reduction with an average bitrate increase of 0.9%.

Multicoreware has the so-called UHDkit https://x265.com/uhdkit-hevc-encoding-library/

There is also a white paper from Vanguard on the topic http://vanguardvideo.com/wp-content/uploads/2015/04/HEVCABRMultistreamWhitepaper.pdf

All in all, not so much details about the technical solutions. Is there some more open-source work on multi-rate out there? Do you know some other sources of information about the topic? My guess is that all major streaming providers are implementing such methods...

[Motenai Yoda]

I'm not sure but how is it different from the analysis-mode of x265?


--analysis-mode <string|int>

Specify whether analysis information of each frame is output by encoder or input for reuse. By reading the analysis data writen by an earlier encode of the same sequence, substantial redundant work may be avoided.

The following data may be stored and reused: I frames - split decisions and luma intra directions of all CUs. P/B frames - motion vectors are dumped at each depth for all CUs.

Values: off(0), save(1): dump analysis data, load(2): read analysis data

[benwaggoner]

Analysis-mode allows much faster encoding for the same frame size at different bitrates, yes.


Sent from my iPad using Tapatalk

[damjeux]

Yes, the implemented method is very similar to analysis-mode. The main difference is that the the reuse of the data stored in analysis-mode is not targeted at different representations, that is, different bitrates/different qualities.

As an example, I tried analysis-mode 1 on an encoding at QP 22, then analysis-mode 2 on encodings at QP 27, 32, and 37. The result is an increased bitrate (on average 5.8% at a given PSNR), and an overall encoding time decreased by 23%, compared to the same four representations without analysis-mode.

On the contrary, if I use the implemented method, e.g. mr-mode 1 on QP 22, then mr-mode 2 on QPs 27, 32, and 37, I get for example 0.0% increased bitrate on average at a given PSNR and an overall encoding time decrease of 10%.

Or if I use mr-mode 1 on CRF 22, then mr-mode 2 on CRF 27, 32, and 37, I get 0.9% average increase in bitrate and an overall encoding time reduction of 19%.
These results are detailed in the blog post I mentioned earlier.

So the benefit of the implement method is that the rate-distortion performance is better than with analysis-mode (in this specific scenario of having multiple representations at different qualities).

[x265_Project]

Hi Damjeux,
I'm sorry I didn't reply when you first informed us of your work on our development mailing list. We have a team that is also working on improving analysis save/load/refine, for a variety of interesting applications (not just multi-bitrate encoding). We would be interested in evaluating your contributions for possible inclusion in x265, but we would first need you to sign our Contributor License Agreement (https://bitbucket.org/multicoreware/x265/downloads/x265ContributorAgreement.pdf).

If you would like to collaborate directly with our team, we would be happy to arrange a phone call to discuss.

Thanks,
Tom

[damjeux]

Thanks for the reply. I've sent you an email.

[kabelbrand]

Analysis-mode allows much faster encoding for the same frame size at different bitrates, yes.
What about different frame sizes? With x264 I'd run a first pass in 1080 and then a second pass in multiple resolutions using the log from the 1080 first pass.
Does that work with x265 as well?

[benwaggoner]

What about different frame sizes? With x264 I'd run a first pass in 1080 and then a second pass in multiple resolutions using the log from the 1080 first pass.
Does that work with x265 as well?
I believe that can work. However, the bigger the delta in frame size for the second pass (in x264 when I've tried this, certainly), the bigger the quality loss in the second pass. If you have a broad range of frame sizes, I've not found that to be a useful technique on net.

Variants of the same frame size have a much smaller quality hit, more than made up for being able to boost up the --preset a notch due to perf savings.