Optimize x265 for AppleTV streaming

[ShortKatz]

I have stored all my DVDs as HEVC file on my NAS to stream it from there to my AppleTV. Are there any option I could add, to optimize my files for streaming? I was thinking of --aud or --repeat-headers or similar. Would this help? I sometimes have the issue, that I see the progress bar for several minutes, until the video starts. I assume, it is collecting some information about the video to set the decoder right for the stream? Are there any options that could help in this situation? I need to say, mostly it starts very fast, but only sometimes it doesn't.

[Blue_MiSfit]

The HEVC decoder on modern Apple devices is extremely good. The Apple TV can handle 4kp60 at up to 150 Mbps (Main10 @ 5.1 High).

I'd suggest using a relatively short keyframe interval (2-4 seconds) for good seeking performance, and VBV to at least conform with the above. Other than that you should be able to use whatever settings you want.

[Boulder]

Is that AppleTV 4K or the previous generation?

[ShortKatz]

The HEVC decoder on modern Apple devices is extremely good. The Apple TV can handle 4kp60 at up to 150 Mbps (Main10 @ 5.1 High).

I'd suggest using a relatively short keyframe interval (2-4 seconds) for good seeking performance, and VBV to at least conform with the above. Other than that you should be able to use whatever settings you want.

Thanks. So, you mean --keyint=2 or is it --min-keyint I have to use?
Apple tells about keyframe "1.13. Key frames (IDRs) SHOULD be present every two seconds."
https://developer.apple.com/documentation/http_live_streaming/hls_authoring_specification_for_apple_devices

Is that AppleTV 4K or the previous generation?

The most recent 4k one.

[Boulder]

It would be --keyint 48 to make the max keyint interval 2 seconds (~24 fps video). I would set min-keyint to 5 or something so it won't place adjacent ones if there are sudden flashes in the video.

Your device should definitely be powerful enough for playback. Can you provide the MediaInfo output of a normally working and the other case? Or does it occur randomly? When I had an AppleTV 4K, I didn't have any such issues with MrMC (based on Kodi).

[ShortKatz]

It would be --keyint 48 to make the max keyint interval 2 seconds (~24 fps video). I would set min-keyint to 5 or something so it won't place adjacent ones if there are sudden flashes in the video.
Ah yes, now I see, makes sense, yes. Thanks.


Your device should definitely be powerful enough for playback. Can you provide the MediaInfo output of a normally working and the other case? Or does it occur randomly? When I had an AppleTV 4K, I didn't have any such issues with MrMC (based on Kodi).

Think is, I don't remember which one played nicely and which one did not. And I can't test at the moment, because I've sold my LCD beamer recently. But I will buy a new one, and then I can test and tell you.

[ShortKatz]

If I don't specify --keyint and --min-keyint, x265 gives me an info line:
x265 [info]: Keyframe min / max / scenecut / bias : 24 / 240 / 40 / 5.00

I was wondering, why this line disappears completely, if I specify --keyint and --min-keyint?

In addition, is it in my case recommended to disable scene cut detection to avoid unexpected key frames? Like described here: https://github.com/matmoi/create-DASH-HLS

[Boulder]

If I don't specify --keyint and --min-keyint, x265 gives me an info line:
x265 [info]: Keyframe min / max / scenecut / bias : 24 / 240 / 40 / 5.00

I was wondering, why this line disappears completely, if I specify --keyint and --min-keyint?

That's odd. I set both and have this line:
x265 [info]: Keyframe min / max / scenecut / bias : 5 / 480 / 40 / 5.00

In addition, is it in my case recommended to disable scene cut detection to avoid unexpected key frames? Like described here: https://github.com/matmoi/create-DASH-HLS
I would still keep it enabled. Keyframes are good for quality.

[excellentswordfight]

If I don't specify --keyint and --min-keyint, x265 gives me an info line:
x265 [info]: Keyframe min / max / scenecut / bias : 24 / 240 / 40 / 5.00

I was wondering, why this line disappears completely, if I specify --keyint and --min-keyint?

In addition, is it in my case recommended to disable scene cut detection to avoid unexpected key frames? Like described here: https://github.com/matmoi/create-DASH-HLS
You referenced the HLS specifications a few times, just to make it clear, this is whats recommended for HLS-streaming, and the requirements there differ from local network playback as the settings there needs to accommodate for adaptive streaming for multiple encodes.

For your use case, I would just set keyint (fps*4), min-keyint (fps) and specify level-idc (that enables some vbv limitations that can help with bitrate spikes that can create buffer issues).

[ShortKatz]

I would still keep it enabled. Keyframes are good for quality.

OK, I will keep it.


You referenced the HLS specifications a few times, just to make it clear, this is whats recommended for HLS-streaming, and the requirements there differ from local network playback as the settings there needs to accommodate for adaptive streaming for multiple encodes.

For your use case, I would just set keyint (fps*4), min-keyint (fps) and specify level-idc (that enables some vbv limitations that can help with bitrate spikes that can create buffer issues).

My thought was if it is good for HLS it will also be good for AppleTV streaming. Because I could not find any specific specifications for AppleTV streaming. Thanks, I will try your suggestions.

[Boulder]

By streaming, do you mean decoding the content on the NAS and streaming that or the normal way of working which is just simply transferring the file over the network and decoding it on the ATV4K?

In the latter case, you can encode the files in almost any way you want. I never had any problems with decoding any x264 encodes.

[ShortKatz]

By streaming, do you mean decoding the content on the NAS and streaming that or the normal way of working which is just simply transferring the file over the network and decoding it on the ATV4K?

In the latter case, you can encode the files in almost any way you want. I never had any problems with decoding any x264 encodes.

The latter case. The NAS is just for storing. For decoding I use the default Apple media player on the AppleTV 4k or Infuse. I use x265 encodes, not x264.

[Boulder]

ATV4K should use hardware decoding for HEVC. In that sense, your problem seems related to network issues. Have you checked the NAS web interface to see what kind of rate it transfers when you start playback?

[ShortKatz]

No, I didn't. Good point. I will check that.

[benwaggoner]

My thought was if it is good for HLS it will also be good for AppleTV streaming. Because I could not find any specific specifications for AppleTV streaming. Thanks, I will try your suggestions.
Apple's HLS encoding guidelines have never been more than guidelines, and specify lots of things that aren't needed, and leave out other things that are. In some cases, they're actually impossible, like the long-lasting recommendation for 10 second fragments with keyframes every 3 seconds, for HLS M2TS. 10 is not mod3!

I don't know that a 2 second max GOP duration is really needed, but would work fine (I'd probably us 4 or 5 sec max). And I'd definitely leave scene cut detection ON, since we generally want random access to go to the first frame of a given shot. Open GOP would be fine for this use case as well.

Nice thing about LAN playback is that one can have a lot more flexibility on peak and average bitrates than having to go over the public internet, which can get traded for faster encoding and less tuning.

[RanmaCanada]

Sadly Apple does not make it easy to watch your own content. But since you are using INFUSE, you will not have any problems as it will handle pretty much everything perfectly, with full hardware playback. Apple devices are great for their eco-system, and nothing more. You really should move to a Shield/Android device if you want a hassle free experience.

[Blue_MiSfit]

It's all HLS, AFAIK. The streaming server probably does a dynamic packaging operation.

Apple's HLS encoding guidelines have never been more than guidelines, and specify lots of things that aren't needed, and leave out other things that are. In some cases, they're actually impossible, like the long-lasting recommendation for 10 second fragments with keyframes every 3 seconds, for HLS M2TS. 10 is not mod3!

The new authoring guidelines make a lot more sense :) https://developer.apple.com/documentation/http_live_streaming/hls_authoring_specification_for_apple_devices

They still recommend a 2 second GOP, but now recommend a 6 second segment size.

For production HLS streaming I generally use a fixed 2 second GOP without scenecut, and generally stick with 2 second segments. It kills my soul, but this simple structure is very compatible with different packagers and players. I've been meaning to spend some time revisiting this (as it's been years since I've really looked at it). Bigger fish to fry these days.

x265 had some rate control issues with non-IDR I frames near the end of chunk boundaries, and disabling scenecut worked around them. I think this has been fixed too...

[ShortKatz]

Thanks for all your tips. I have checked, how Apple is setting the keyframe interval if exporting a HEVC file with Apple Compressor for Apple devices. It seems Apple is using a very short keyframe interval. I exported a 8-bit and 10-bit file using the "Apple device" preset and checked the GOP with iframe-probe.py


8-bit HEVC:
Stream #0:1(und): Video: hevc (Main) (hvc1 / 0x31637668), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 7489 kb/s, 23.98 fps, 23.98 tbr, 24k tbn, 24k tbc (default)
Metadata:
creation_time : 2020-11-20T21:18:20.000000Z
handler_name : Core Media Video
GOP: IBBBPBBBPBBBPBBBPBBBPBBBPBBBPP 30 CLOSED
GOP: IBBBPBBBPBBBPBBBPBBBPBBBPBBBPP 30 CLOSED
GOP: IBBBPBBBPBBBPBBBPBBBPBBBPBBBPP 30 CLOSED
GOP: IBBBPBBBPBBBPBBBPBBBPBBBPBBBPP 30 CLOSED
GOP: IBBBPBBBPBBBPBBBPBBBPBBBPBBBPP 30 CLOSED
GOP: IBBBPBBBPBBBPBBBPBBBPBBBPBBBPP 30 CLOSED
GOP: IBBBPBBBPBBBPBBBPBBBPBBBPBBP 28 CLOSED



10-bit HEVC:
Stream #0:1(und): Video: hevc (Main 10) (hvc1 / 0x31637668), yuv420p10le(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 7509 kb/s, 23.98 fps, 23.98 tbr, 24k tbn, 24k tbc (default)
Metadata:
creation_time : 2020-11-20T20:34:44.000000Z
handler_name : Core Media Video
GOP: IBBBBBBBBBBBBBBBBBBBBBBBBBBBBB 30 CLOSED
GOP: IBBBBBBBBBBBBBBBBBBBBBBBBBBBBB 30 CLOSED
GOP: IBBBBBBBBBBBBBBBBBBBBBBBBBBBBB 30 CLOSED
GOP: IBBBBBBBBBBBBBBBBBBBBBBBBBBBBB 30 CLOSED
GOP: IBBBBBBBBBBBBBBBBBBBBBBBBBBBBB 30 CLOSED
GOP: IBBBBBBBBBBBBBBBBBBBBBBBBBBBBB 30 CLOSED
GOP: IBBBBBBBBBBBBBBBBBBBBBBBBBBBBB 30 CLOSED
GOP: IBBBBBBBBBBBBBBBBBBBBBBBBBBBBB 30 CLOSED
GOP: IBBBBBBBBBBBBBBBBBBBBBBBBBBBBB 30 CLOSED
GOP: IBBBBBBBBBBBBBBBBBBBBBBBBBBBBB 30 CLOSED
GOP: IBBBBBBBBBBBBBBBBBBBBBBBBBBB 28 CLOSED


Why does the 10-bit HEVC contain only b-frames?

[Blue_MiSfit]

That is indeed odd. I can personally confirm that P frames work fine for 10 bit HEVC on all Apple devices :)

Regarding the short keyframe interval, perhaps this is just the default for excellent scrub performance?

Also, how did you get that neat visualization of the GOP structure? That's handy.

[ShortKatz]

Also, how did you get that neat visualization of the GOP structure? That's handy.

I used iframe-probe.py from here:
https://gist.github.com/alastairmccormack/7041ee993adb5c911f90

Just do
python iframe-probe.py /path/to/movie


I also like this one https://lulebo.github.io
But sadly this does not work for HEVC.

[SeeMoreDigital]

I have stored all my DVDs as HEVC file on my NAS to stream it from there to my AppleTV....HEVC in which container?

Doesn't the AppleTV 4K support the MPEG-2 video format?

[ShortKatz]

HEVC in which container?

m4v



Doesn't the AppleTV 4K support the MPEG-2 video format?
Not natively, but Infuse does. You are going to say, why not save the original rip on the NAS? I also did that, but the files are very huge, they can not be played natively on AppleTV and it takes several minutes with Infuse until the movie starts.

[SeeMoreDigital]

Hmmm....

Apple devices are far too anal for my liking :eek:

[Boulder]

Hmmm....

Apple devices are far too anal for my liking :eek:

Tell me about it.. my last two straws were that they removed support for bitstreaming so bye bye Dolby Atmos, and true 24p (=24 fps) content is output like 23.976 fps content is :eek:

[RanmaCanada]

Hmmm....

Apple devices are far too anal for my liking :eek:

And that is why I suggested a Shield. Apple devices are great if you only want to play in their world. As soon as you try to do anything outside of it, everything comes crashing down, hard. And yet people still flock to them because they believe they are the best. INFUSE does a great job of fixing these issues, but it's not a cheap fix, and it's one that should not be required.

[rwill]

Why does the 10-bit HEVC contain only b-frames?

The question is rather why they are using P-Frames for 8 bit when they could use B as that would be more efficient.

[SeeMoreDigital]

And that is why I suggested a Shield. Apple devices are great if you only want to play in their world. As soon as you try to do anything outside of it, everything comes crashing down, hard....The media players in most smart TV's can play a wide variety of video and audio formats, and container types too. They do however have very poor looking user interfaces...

[Boulder]

The media players in most smart TV's can play a wide variety of video and audio formats, and container types too. They do however have very poor looking user interfaces...

They are often quite horrible. The standard player in my Sony doesn't support loading files over a network share so any DoVi stuff has to be played from a USB drive.

For a generic media player, I currently suggest an ODROID N2+ with CoreELEC. Sadly DoVi is not supported by Kodi yet, but maybe in the future once the SoCs start doing that in a larger scale.

[RanmaCanada]

The media players in most smart TV's can play a wide variety of video and audio formats, and container types too. They do however have very poor looking user interfaces...

Most TV's still can't handle TRUEHD and subtitles. OMG subtitle support is so lacking. SRT works most of the time, but ASS, vobsub, PGS? HAHAHA. Apple TV is in the same boat, as it only supports SDH and captions I believe.

[Blue_MiSfit]

"SDH / Captions" are content types, not file formats.

The Apple ecosystem (namely first-class delivery via HLS) supports delivery of timed text in 3 ways:

1) Embedded 608 closed captions (often used in live feeds repurposed from broadcast)

2) WebVTT

3) ISMC1 Text profile TTML


Most VOD content, from what I've seen and personally authored uses WebVTT due to simplicity. IMSC1 is quite powerful, but a bit more complex to author. Also from what I've seen, (1) is considered "captions", and (2) / (3) are considered "subtitles".

When it comes to subtitles, you can of course tag tracks with languages, but can also set the "forced" attribute to differentiate forced subtitles from regular subtitles / SDH.

[foxyshadis]

The only Dolby Vision streaming device I know of is the 2020 Amazon Fire Cube, and who knows if that's hardware or not, they never tell us. Likewise whether or not it supports DoVi in external libraries instead of just Netflix/Prime/etc.

[benwaggoner]

The only Dolby Vision streaming device I know of is the 2020 Amazon Fire Cube, and who knows if that's hardware or not, they never tell us. Likewise whether or not it supports DoVi in external libraries instead of just Netflix/Prime/etc.
What do you mean by "if that's hardware or not?"

I don't know of anything that does DoVi in user mode processing from an app, since DoVi involves a LOT of processing that has to happen in protected memory for DRM reasons, so it's handled at the OS level. Many SoCs do have HW support for Dolby Vision processing, but with sufficiently powerful CPU + GPU it could all be done in software.

I actually had a prototype DoVi implementation working on a quad-core 2015 ARM tablet that could do 1080p24, albeit without DRM support.

The AppleTV also supports Dolby Vision.

[Blue_MiSfit]

The only Dolby Vision streaming device I know of is the 2020 Amazon Fire Cube, and who knows if that's hardware or not, they never tell us. Likewise whether or not it supports DoVi in external libraries instead of just Netflix/Prime/etc.


The AppleTV also supports Dolby Vision.

So does the latest and greatest Roku Ultra. And of course the Xbox. And Chromecast Ultra.

IIRC you can implement Dolby Vision playback in ExoPlayer on the new FireTV just like you would on an Android TV device (Sony, nVidia Shield, etc) streaming either DASH or HLS. You may need to use Amazon's fork of ExoPlayer as there were at least historically some customizations there for the Fire platform vs Android.

As Ben said, the system handles all the fancy stuff for you. Dolby Vision via HEVC has its own codec IDs (dvhe, dvh1) so from an application standpoint not much special generally has to be done.

Apple TV does a very nice job with Dolby Vision playback, and is my personal favorite streaming device for any HDR media (tho the remote is rage inducing at times). They have a robust hardware decoder and a powerful SOC resulting in a very smooth UI. They also have large buffers, and good ABR performance. They can even do a conversion from Profile 5 DoVi into HDR10 in real-time if you're using a TV with only HDR10 support. It's pretty clever - AFAIK they do IPT -> YCbCr without any tone mapping, and use the lowest level DoVi metadata to synthesize static HDR10 ST2086 metadata