iPad 3/HD/2012/"New" h.264 encoder settings

[HMJ]

I have searched the forums, particularly for the last two weeks, and have found nothing about this. OK, the new iPad is out and it's got this fabulous high-res display. Unfortunately this is both a blessing and a curse. Many people, myself included, have had problems playing HD 720p, forget 1080p, mkv and avi files smoothly and without audio sync problems. The two players of choice seem to be GoodPlayer and AVPlayerHD. Both have shown this problem. I have GoodPlayer so I'm trying to work with tech support on this issue.

So far their response to the issue is that the iPad 3 performs worse than the iPad 2:

We just got our new iPad . In our performance testing, new iPad is slower than the iPad 2. Please do not expect too high of iPad 3.
Now for GoodPlayer performance:
iPad 2 > new iPad (iPad 3) > iPhone 4S > iPad 1 >iPhone 4 > iPhone 3GS

They have given some very rough guidelines on encoding settings that are supposed to work but no specific recommendations.

For a better performance: no B frames is good; less reference frames is good; no CABAC is good

Transmuxing into an mp4 container certainly improves results dramatically but I am trying to avoid having to do that--my sole reason for buying the player in the first place.

By now (two weeks after iPad 3 launch) someone here has certainly encountered the same problem and, I am hoping, come up with at least a partial solution.

What I am looking for is recommendations for x264 parameter settings to encode HD sources to an mkv or mp4 container at 720p frame size that play properly on either of these iPad players. 1080p settings would also be nice but not necessary. I am primarily looking for video settings since audio settings are obvious.

Apple gives some vague specs on supported video formats but, clearly, there are encoding parameters that would optimize playback.

Video formats supported: H.264 video up to 1080p, 30 frames per second, High Profile level 4.1 with AAC-LC audio up to 160 Kbps, 48kHz, stereo audio in .m4v, .mp4, and .mov file formats

[nm]

The hardware is capable of decoding High Profile 1080p30 video at L4.1 with B-frames (B-pyramid is probably supported too), CABAC and at least as many reference frames as you can fit to L4.1. Most other things shouldn't make much of a difference, but you may want to use Blu-ray-compatible settings to play it safe.

If the same video plays ok when muxed to MP4, the problem is not in the H.264 stream but in your player. I doubt there's much you can do about it by changing encoding settings.

[nibus]

In my experience, matroska support on 3rd party iPad players is pretty bad. I only have an iPad 1 but I've encountered lots of lag, audio sync issues, and other problems. However when muxed to mp4 the iPad can play videos back with hardware acceleration flawlessly. I know it's a lot of work, but I would suggest re-muxing to mp4. The muxer of choice for this is subler which unfortunately only runs on Mac OS. But Yamb also works as does anything with mp4box.

[HMJ]

Originally Posted by nm

The hardware is capable of decoding High Profile 1080p30 video at L4.1 with B-frames (B-pyramid is probably supported too), CABAC and at least as many reference frames as you can fit to L4.1. Most other things shouldn't make much of a difference, but you may want to use Blu-ray-compatible settings to play it safe.

As for settings, the developers don't recommend any High profile settings:

Here are some brief suggests (again, this is just for quick start, please check related encoding information carefully, there are too many parameters out there):
1. CABAC , hard
2. too many reference frames , hard
3. High profile, hard

If the same video plays ok when muxed to MP4, the problem is not in the H.264 stream but in your player. I doubt there's much you can do about it by changing encoding settings.

The problem is definitely with the player. It clearly can't handle advanced settings or, presumably, very high bitrates.

The developer is very vague about just what it can handle:

About HD (High Definition) playback support::
For iPad 2, part of 720P(1280x720) MKV/H.264/AVC playback supported,
please note that not all the 720P MKV/H.264/AVC files can be played
smoothly, it depends on the specific video encoding parameters.

I'd really rather not have to transmux everything and, unfortunately, this is the highest rated mkv player, so I would like to spend some time on getting it to work. I sure would appreciate any input from others who have tackled this problem. So would the general iPad community. This is a persistent problem.

[Dark Shikari]

That doesn't look like an official Apple post. Anyone who pays $99 can be a Mac developer; that doesn't mean they know what they're talking about.

The iPad 2 almost surely has a hardware decoder just like all of the previous Apple mobile devices.

[HMJ]

Originally Posted by nibus

In my experience, matroska support on 3rd party iPad players is pretty bad. I only have an iPad 1 but I've encountered lots of lag, audio sync issues, and other problems. However when muxed to mp4 the iPad can play videos back with hardware acceleration flawlessly.

Yes, that is the general consensus unfortunately.

I know it's a lot of work, but I would suggest re-muxing to mp4. The muxer of choice for this is subler which unfortunately only runs on Mac OS. But Yamb also works as does anything with mp4box.

I'll have no choice if I can't solve this, but, for now, I'd rather avoid this laborious workaround. Apple really sucks about its file support policies. I refuse to buy any media from them but they obviously have a large enough customer base that does. THAT's the real problem.

[shroomM]

What about VLC for the iPad, what's the status on that?

[J_Darnley]

Wasn't it "not happening" due to the problems obeying the GPL when you use Apple's store?

[HMJ]

Originally Posted by Dark Shikari

That doesn't look like an official Apple post. Anyone who pays $99 can be a Mac developer; that doesn't mean they know what they're talking about.

It is a post by the app developer. Considering that they got an app approved by Apple's draconian submittal process, for an app that allows users to bypass the iTunes store for their media, it sounds like maybe they do know something. It is possible that it got approved because its performance was no threat to the built-in player.

The iPad 2 almost surely has a hardware decoder just like all of the previous Apple mobile devices.

Of course it has and so does the iPad 3 which we're talking about here. It is only programmed to decode mp4 and mov container formats though. The developers mentioned that they have code that performs better but that it accesses resources that Apple forbids apps to access so they can't incorporate it. Face it, Apple sucks. But the hardware is unbelievable. That's why I am pursuing this.

[HMJ]

Originally Posted by shroomM

What about VLC for the iPad, what's the status on that?

It was pulled from the app store for some alleged violation. I had it on my iPod and iPhone but it really sucked out loud. The UI was miserable and playback was poor on anything other than mp4 and avi. There is a newer version available if you want to jailbreak but it's not worth the trouble for me. This app is way better.

At least the developers are responsive. Because of the new audio sync delay problem with the upgrade to iOS 5.1, they have included a delay button on the latest rev, so I have to give them some credit for trying. But in the final analysis, there is no "try", there is only "do".

[nm]

Originally Posted by HMJ

Of course it has and so does the iPad 3 which we're talking about here. It is only programmed to decode mp4 and mov container formats though. The developers mentioned that they have code that performs better but that it accesses resources that Apple forbids apps to access so they can't incorporate it.

And did they say what those resources are? If the problem really is that their app can't use hardware decoding, I'd forget about it. Software decoding won't be fast enough, drains the battery and heats up the device.

If you insist, try 720p with CAVLC instead of CABAC and turn deblocking off in the encoder. With x264 you can simply use --tune fastdecode, which also turns off weightb and weightp. 1080p probably can't be played in real time.

But I don't see how re-encoding everything to this format would be easier than remuxing to MP4?


Edit: The website of the app says that they use the iPod plugin for hardware-accelerated playback of MP4/MOV. So apparently they use pure software decoding for other formats. I wonder if some Apple video component would play a realtime streaming format such as TS, in which case it might be possible to remux MKV on the fly within the device.

[HMJ]

Originally Posted by nm

And did they say what those resources are? If the problem really is that their app can't use hardware decoding, I'd forget about it. Software decoding won't be fast enough, drains the battery and heats up the device.

Well of course it's a given that they have to use software decoding since iOS only supports GPU decoding of mp4/mov formats. But that doesn't mean it can't be fast enough if the encoding parameters are optimized. Even on mkvs that it struggles with, the battery barely gets warm after hours of play and the drain is negligible. No exaggeration.

As for the resources:

Originally Posted by hustmobile

The MP4/M4V/MOV container restriction is from Apple official API. So for official app , the MKV can only use software decoding.
In fact, there is non-official private API can decode H.264 frame (then 1080P is possible here), however, if an app need go to App Store, you cannot use non-official private API.
So the restriction is just there, it is not iOS cannot do it.

Originally Posted by nm

If you insist, try 720p with CAVLC instead of CABAC and turn deblocking off in the encoder. With x264 you can simply use --tune fastdecode, which also turns off weightb and weightp. 1080p probably can't be played in real time.

Well that's a start anyway. I tried one 1080p mkv and it just shows still images with audio.

Originally Posted by nm

But I don't see how re-encoding everything to this format would be easier than remuxing to MP4?

No it absolutely wouldn't. I just want to know encoding parameters so I can occasionally re-encode my own BDs or know what to look for in already encoded mkvs. Otherwise transmuxing to mp4 would be the best route, one I'd like to avoid however.

Originally Posted by nm

Edit: The website of the app says that they use the iPod plugin for hardware-accelerated playback of MP4/MOV. So apparently they use pure software decoding for other formats. I wonder if some Apple video component would play a realtime streaming format such as TS, in which case it might be possible to remux MKV on the fly within the device.

Here is their take on that very idea:

Originally Posted by dbknightx

Do you think it would be possible for an app (like GoodPlayer) to do an on-the-fly remuxing from MKV to MOV containers, and then feed that to the player, thus allowing it to use GPU acceleration?

Originally Posted by hustmobile

For the on-the-fly remuxing idea, technically it should be possible. However, it is not just remuxing, sometimes re-encoding is needed. For your blu-ray case, the audio needs to be re-encode to AAC.
Although the idea is possible, but it is somewhat hard to be implemented.

[kieranrk]

Where's their source code?

[CruNcher]

It's the same game with Sony only specific container formats supported for Hardware acceleration most compatible would be .mp4 and of course .mov on Apple devices also the PSP had to be broken to use the Hardware acceleration for other Formats not sure how it stands with the Vita currently but i guess Sony is still on the same Trip

[nm]

Originally Posted by HMJ

Well of course it's a given that they have to use software decoding since iOS only supports GPU decoding of mp4/mov formats.

As your quote said, it's only limitation of the official media playback framework, which is their only option to use hardware decoding and be allowed at App Store. Besides, the official APIs probably do support MPEG-TS too, at least in form of HTTP Live Streaming.

Originally Posted by HMJ

But that doesn't mean it can't be fast enough if the encoding parameters are optimized.

You can't optimize encoding parameters significantly more than what --tune fastdecode does. Software decoding of 720p H.264 video at low to medium bitrates is possible on A4 and A5X, but 1080p is not.

For software decoding of 1080p the only option is to use MPEG-4 ASP or MPEG-2 instead of H.264, and I have some doubts that even their MPEG-2 decoder would be fast enough.

Originally Posted by HMJ

Even on mkvs that it struggles with, the battery barely gets warm after hours of play and the drain is negligible. No exaggeration.

Yep, checked the numbers and apparently the display draws ten times more power than the CPU at full load.

Originally Posted by HMJ

No it absolutely wouldn't. I just want to know encoding parameters so I can occasionally re-encode my own BDs or know what to look for in already encoded mkvs.

You won't find already encoded files with these parameters because they suck. Personally I'd re-encode those Blu-rays to a format that is supported by Apple's player and avoid wasting disk space with poor encoding settings.

[CruNcher]

1080p isn't possible ? i doubt that as Dark said its a own Decoder core apart from the ARM cpu, and these days 1080p is pretty standard even for all the Chinese Chips
Tegra 3 already broke the 2K barrier

[nm]

Originally Posted by CruNcher

1080p isn't possible ?

Software decoding of 1080p. HMJ wants to use software decoding to be able to play MKV files.

[CruNcher]

ah ok sorry yeah that could be a problem even with lowest possible complexity i guess on the A5

[Kisa_AG]

Hello Gentlemens!

I also trying to find good encoding settings to encode 720p25 and 1080p25 for IPAD 3. But I decided to use MP4 as a container.

I started with the following string:

x264 --bitrate 7000 --preset veryslow --tune film --vbv-maxrate 10000 --vbv-bufsize 10000 --level 4.1 --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1

and it seems to play smoothly on IPAD. But during play on IPAD I see blocking on some "fast" frames.

What options should I add to this string to make the encoding more adequate?

Is it make sence to use --nal-hrd vbr, for example, or --bluray-compat, or to use --keyint parameter, or --open-gop?

The second task for me is to add 720p50 to IPAD, but iTunes doesn't allow me to import such mp4 to IPAD. But I know that IPAD CAN play such files because when I added 720p50 video from GoPro camera thrue Camera Connection Kit, IPAD play it smoothly without any problems.
So it would be great if you have any suggestions for encoding string for 720p50.

[nm]

Originally Posted by Kisa_AG

I started with the following string:

x264 --bitrate 7000 --preset veryslow --tune film --vbv-maxrate 10000 --vbv-bufsize 10000 --level 4.1 --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1

and it seems to play smoothly on IPAD. But during play on IPAD I see blocking on some "fast" frames.

What options should I add to this string to make the encoding more adequate?

Use CRF encoding instead of 1-pass ABR! Replace --bitrate 7000 with --crf 21.
Higher VBV settings might also be in order. No need to limit the bitrate too much.

Originally Posted by Kisa_AG

Is it make sence to use --nal-hrd vbr, for example, or --bluray-compat, or to use --keyint parameter, or --open-gop?

No need for those. Default keyint is ok.

Originally Posted by Kisa_AG

The second task for me is to add 720p50 to IPAD, but iTunes doesn't allow me to import such mp4 to IPAD. But I know that IPAD CAN play such files because when I added 720p50 video from GoPro camera thrue Camera Connection Kit, IPAD play it smoothly without any problems.

Just don't use iTunes for transferring the file.

Originally Posted by Kisa_AG

So it would be great if you have any suggestions for encoding string for 720p50.

Use the same command line as for 1080p encoding. Keyint could be increased to 500 for the same 10 second seek accuracy.