How to keep compatibility with hardware players?

[W3ird_N3rd]

I encode some h.264 movies from time to time, it would be nice if I could keep them compatible with hardware players. Like videocard-hardware decoding, mediaplayers in/for TVs, maybe even phones. You never know what I want to play the file on in the future.

Obviously I don't want to sacrifice too much quality, but leaving out some obscure options that don't improve the quality much anyway is ok.

I mostly use mencoder or avidemux. What options should I use, or stay far from? And is there any way to check if my file is compatible, without actually buying hardware decoders?

Or is this all obsolete today? In the old days, you just made sure you didn't use Qpel and GMC on your XviD and you'd be fine. With h.264, things are obviously different, but I'm not sure in what way.

It would also be nice to know what settings the "scene" uses, as their files generally seem to play on anything.

(I did search but could not quite find what I was looking for)

[nurbs]

You should stick to a level, apart from some exotic things that you could never turn on without using obscure options that's the only thing that matters for compatibility. Level 4.1 works on most non-mobile hardware and some newer cell phones. The AppleTV is a notable exception there. Level 3.1 works on even more including AppleTV, iPhone (3GS upward IIRC) and more other phones. You'll be restricted to 720p using that. You can read about the level restrictions on Wikipedia. The restrictions are largely on resolution, framerate and number of reference frames. It also restricts bitrate, but that won't be a real limitation in most backup scenarios. Just make sure you set VBV maxrate and bufsize accordingly to be on the safe side.

The scene rules are somewhat decent these days, but there is no reason to stick to them if your main goal is hardware compatibility, plus they are bordering on Rule 6 territory around here. The other stuff in there is just about encoding settings and filesizes. For the former you have the inbuilt x264 presets and tunings and I don't see why anyone would stick to the scene sizes unless they were targeting CD or DVD size anyway. If you don't do that you should just find a combination of --preset, --tune and --crf you are confortable with and not worry about sizes.

A sample command line for x264 itself that works on many devices with 720p video would be:
x264 --preset slow --tune film --crf 21 --vbv-maxrate 17500 --vbv-bufsize 17500 --level 3.1 -o output.mkv input.avs

You'll have to find a way to adapt that for mencoder.

[CarlEdman]

I agree generally, with @nurbs, except I've had a fair amount of trouble with profile 3.1 with Apple Devices (including the current iPhone and iPad), but not with profile 3.0. As that matters a great deal to my family, I've gone back to encoding my standard-def content to level 3.0.

[nurbs]

I don't actually own any Apple devices, I just mentioned them because they are pretty popular. Level 3.1 is what Handbrake uses for the newer ones, maybe with some stricter VBV restrictions and possibly turning off weightp, I haven't checked. Anyway, since Handbrake is popular I'm sure their presets work so it's probably a good idea to if that's your target.
Personally I use 3.0 for my SD stuff and 3.1 for Blu Rays that I downscale to 720p.

[W3ird_N3rd]

Thanks nurbs and CarlEdman!

I didn't mean to break rule 6, this is not about illegal content, I just wondered if the settings from the "scene" (who most probably have been spending quite some time finding good and compatible settings) might be useful for me.

So as long as I get something from x264 like:

x264 [info]: profile High, level 3.0

Up to 4.1 (or 3.1 for Apple) I should be good, right? I've now made some encoding lines for SD at 3.0, 720p on 3.1 and 1080p at 4.1. Although I likely won't be using 1080p as that'll take forever on my X2 5600+ 2.9Ghz.

CarlEdman, thanks for pointing out problems with Apple and 3.1. I'm not really planning on buying Apple, but it would be nice to stay compatible, just in case. Looking at the handbrake profiles, many (especially slightly older) Apple devices don't even support CABAC. Well I'm not going to turn that off.

The iPhone4/iPad profile does not have any special options. In fact, it passes no extra options whatsoever to x264. Only reference frames and B-frames are both set to 3, that could be either because that's the highest found in any Handbrake profile, or because iPad/iPhone4 supports no more.

I'll keep the SD-stuff to 3.0, that's what I'll be encoding mostly for the forseeable future.

I've made a 3.1 sample, this should play on an iPhone4/iPad/AppleTV2: http://www.mediafire.com/?rrmp66hicfnznc3 (don't worry, it's Creative Commons!)

Mencoder lines used for this:

mencoder "input" -o "output" -ovc x264 -x264encopts bitrate=3500:pass=1:subme=3:frameref=1:bframes=3:threads=auto:mixed_refs:b_adapt=2:direct=auto -nosound

mencoder "input" -o "output" -ovc x264 -x264encopts bitrate=3500:pass=2:subme=6:frameref=3:bframes=3:threads=auto:mixed_refs:no-fast-pskip:8x8dct:direct=auto:trellis=1:vbv-maxrate=16500:vbv-bufsize=17000 -nosound

If it doesn't play I'm going to be confused.

[nurbs]

Originally Posted by W3ird_N3rd

I didn't mean to break rule 6, this is not about illegal content, I just wondered if the settings from the "scene" (who most probably have been spending quite some time finding good and compatible settings) might be useful for me.

If you are interested the scene wants level 4.1, settings not faster than --preset slow and some really stupid rules on what CRF you can use, that's it.

[nm]

Originally Posted by W3ird_N3rd

Mencoder lines used for this:

mencoder "input" -o "output" -ovc x264 -x264encopts bitrate=3500:pass=1:subme=3:frameref=1:bframes=3:threads=auto:mixed_refs:b_adapt=2:direct=auto -nosound

mencoder "input" -o "output" -ovc x264 -x264encopts bitrate=3500:pass=2:subme=6:frameref=3:bframes=3:threads=auto:mixed_refs:no-fast-pskip:8x8dct:direct=auto:trellis=1:vbv-maxrate=16500:vbv-bufsize=17000 -nosound

MEncoder supports x264 presets and tunings these days, and it does fast first pass by default.

[W3ird_N3rd]

But how fast? If I lower subme to 1 on the first pass I see a big degradation in picture quality.

I'm also reading 8x8dct would already be standard.

Still, I used to have many more options that were no longer needed.

[nm]

Originally Posted by W3ird_N3rd

But how fast? If I lower subme to 1 on the first pass I see a big degradation in picture quality.

It gets lowered to 2, not 1.

I'm also reading 8x8dct would already be standard.

Yes, x264 encodes in high profile by default.

[W3ird_N3rd]

Originally Posted by nm

It gets lowered to 2, not 1.

I ran a test with subme=2 on the first pass, but it's still slighty blockier than subme=3 on the first pass.

I'm going to stick with subme=3.

[nm]

Originally Posted by W3ird_N3rd

I ran a test with subme=2 on the first pass, but it's still slighty blockier than subme=3 on the first pass.

Did you try using the same parameters (except pass=N) for both passes and let x264 lower the settings for you (with older MEncoder versions, you also need turbo=1, but I'd recommend updating instead)?

If you already happen to use current MEncoder, subme will get lowered to 2 by default when you have pass=1 in the command line. You need to use parameter slow_firstpass to avoid that.

[W3ird_N3rd]

Originally Posted by nm

Did you try using the same parameters (except pass=N) for both passes and let x264 lower the settings for you (with older MEncoder versions, you also need turbo=1, but I'd recommend updating instead)? If you already happen to use current MEncoder, subme will get lowered to 2 by default when you have pass=1 in the command line. You need to use parameter slow_firstpass to avoid that.

Damn, you're right. So where did the extra compression artefacts come from? Maybe from opening the player twice.

I'm really confused. I think I'm going crazy. But that's impossible of course. I already was.