h264enc for Linux

[microchip8]

Have you already tried the two xbox presets available (mx and mxhq) ? They don't work? I don't see anything special in the above settings that'll make the already present xbox presets in the script incompatible. The only thing is that CABAC is disabled while I have it enabled in the xbox presets. Maybe xbox1 can't take CABAC?

[mrplow]

Those are xbox360 presets and to tell you the truth I tried them many revisions ago and many xbmc revisions ago and at the time it would stutter from time to time in high action scenes. But that was a while ago and I could try them again this weekend.
It seems the xbox1 can only handle CABAC if Deblocking is disabled and visa versa. With CABAC on and deblocking off the video is better quality but larger and with CABAC off and deblocking on the quality goes down slightly along with the file size.

[microchip8]

hmm, CABAC is supposed to increase compression more than CAVLC so I'm a bit at a loss why you get smaller file sizes when you disable it. If I add such a preset, then I'll only disable CABAC but leave deblocking on. Can you try the xbox presets again on a short sample and see if you still get problems with them on the xbox1?

[daniel.pool]

Hi,

I have a multi core cpu I was wondering if there is an option I need to be setting to make the encode multi-threaded? I got h264enc off the Karmic repository today (version 9.0.0) and am trying out a test encode but it is currently only using a single core. A quick search of this thread and the man pages hasn't revealed anything to me ... :-(

Thanks in advance.
~Dan

[nm]

First of all, I'd recommend getting MPlayer/MEncoder SVN, built against a recent libx264 version and then the latest h264enc version. Ubuntu provides oldish versions of these programs.

Multithreading should work with those versions too, but maybe froggy1 can guess what's going on.

[microchip8]

the threads=auto option is always used in h264enc so it should work. As nm said, get latest stuff from SVN/git/whatever

[daniel.pool]

Froggy1, nm,

Thanks for the really fast reply guys.

I've got the latest x264 from git, but I'm currently unable to get the latest mplayer from svn, one of the dependencies (I'm using this guide on compiling mplayer-nogui, so maybe I'm building completely the wrong thing) has a broken deb. I've got a bug raised against it, so for now the only way I've been able to get h264enc working is to go with what is coming from rvm's ppa (https://launchpad.net/~rvm/+archive/testing).

I'll try upgrading as soon as a fix is made available for the deb (or someone points out I'm being thick and get pointed to the right guide for compiling on Karmic )

[microchip8]

what is this broken dependency? Just go to mplayer's site, get SVN link, and pull from there. That's what I always to (though I have a script called getprog which accepts params about which software to pull from SVN, something small I've written to east it for me)

[daniel.pool]

Originally Posted by froggy1

what is this broken dependency?

To quote Andrew from the Ubuntu forums:

By default Ubuntu does not offer a particularly rich development environment so we need to download the required 'dev' files which are specific to Intrepid Ibex. (These files are loosely modelled on the results of the command 'sudo apt-get build-dep mplayer-nogui', I have removed some files and added others.)

The dep that was failing me was libopencore-amrnb0, as stated, there is a bug raised against it.

However, as it turns out in researching these links to reply to you (and running the apt-get command again to try and get the error), the dependencies now actually install, which means my compilation should succeed! Huzzah.

So, I'll grab the latest h264enc once I've got the latest mplayer. Will update this with my results. Thanks very much for your help.

Thanks
~Dan

[microchip8]

This is why I don't use Ubuntu! They got to have the most incompetent packagers out there :P
I'm a packager myself but luckily not for Ubuntu...

So then... compile everything, first x264 then mplayer, and see if you get threading problems. Make sure to run h264enc -r to reset its config file

[daniel.pool]

Gaah, it's still only running on a single core

I'm pretty sure that it's not an option I've selected (I've attached my options file just to make sure).

I'm going to call it a day on this, but will look at it again tomorrow evening. I'm guessing the best way would be to call the encoder directly from the command line and see what is happening there. Is it mencoder called, or ffmpeg?

Cheers,

[microchip8]

better post attachments to an external site as it can take here some time before they get accepted by a mod
as for the threading. I don't have problems, neither other users who have reported that it works

[kripz]

Im getting errors after updating h264enc, mencoder, mplayer and ffmpeg.

Starting to encode in: 5 4 3 2 1
MPlayer SVN-r29809 (C) 2000-2009 MPlayer Team
Option x264encopts: Unknown suboption weightp
success: format: 0 data: 0x0 - 0x93a0a800
TS file format detected.
VIDEO VC1(pid=4113) AUDIO TRUEHD(pid=4352) NO SUBS (yet)! PROGRAM N. 1
Searching for VC1 sequence header... found
VIDEO: VC-1 1920x1080, 23.976 fps, header len: 30
[V] filefmt:29 fourcc:0x31435657 size:1920x1080 fps:23.976 ftime:=0.0417
==========================================================================
Opening audio decoder: [ffmpeg] FFmpeg/libavcodec audio decoders
Unknown/missing audio format -> no sound
ADecoder init failed
Cannot find codec for audio format 0x44485254.
==========================================================================
Opening video filter: [expand osd=1]
Expand: -1 x -1, -1 ; -1, osd: 1, aspect: 0.000000, round: 0
Opening video filter: [harddup]
Opening video filter: [softskip]
==========================================================================
Requested video codec family [wmvvc1dmo] (vfm=dmo) not available.
Enable it at compilation.
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
Selected video codec: [ffvc1] vfm: ffmpeg (FFmpeg WVC1)
==========================================================================
Movie-Aspect is 1.78:1 - prescaling to correct movie aspect.
FATAL: Cannot initialize video driver.

Exiting...
-> MEncoder has exited with a non-zero value!
-> Exiting in function: mencoder_exit()

Trying to convert a m2ts bluray rip ripped by anydvd. Ive tried AC3 and ACC, no luck.

[microchip8]

@kripz

Look at the top, your libx264 is not new enough as it's missing weightp option. Pull latest x264 from git and compile it, then you may need to recompile mplayer and make sure it's linked to it, not to another older libx264 if you have one on your system

[WalterK]

yeah folks dont forget to keep x264 updated as well as mencoder and mplayer.

[kripz]

lol i totally missed that line

edit: those patches on your website are very outdated?

edit2: Doesnt look right...

Starting to encode in: 5 4 3 2 1
MEncoder SVN-r29934-4.3.2 (C) 2000-2009 MPlayer Team
success: format: 0 data: 0x0 - 0x93a0a800
TS file format detected.
VIDEO VC1(pid=4113) AUDIO TRUEHD(pid=4352) NO SUBS (yet)! PROGRAM N. 1
Searching for VC1 sequence header... found
VIDEO: VC-1 1920x1080, 23.976 fps, header len: 30
[V] filefmt:29 fourcc:0x31435657 size:1920x1080 fps:23.976 ftime:=0.0417
==========================================================================
Opening audio decoder: [ffmpeg] FFmpeg/libavcodec audio decoders
Unknown/missing audio format -> no sound
ADecoder init failed
Cannot find codec for audio format 0x44485254.
==========================================================================
Opening video filter: [expand osd=1]
Expand: -1 x -1, -1 ; -1, osd: 1, aspect: 0.000000, round: 0
Opening video filter: [harddup]
Opening video filter: [softskip]
==========================================================================
Requested video codec family [wmvvc1dmo] (vfm=dmo) not available.
Enable it at compilation.
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
Selected video codec: [ffvc1] vfm: ffmpeg (FFmpeg WVC1)
==========================================================================
Movie-Aspect is 1.78:1 - prescaling to correct movie aspect.
x264 [warning]: NAL HRD parameters require VBV max bitrate and buffer size to be specified
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast FastShuffle SSEMisalign LZCNT
x264 [info]: profile Main, level 4.1
Pos: 0.0s 1f ( 0%) 0.00fps Trem: 0min 0mb A-V:0.000 [0:0]
1 duplicate frame(s)!
Writing header...7f ( 0%) 18.56fps Trem: 0min 0mb A-V:0.000 [0:0]
ODML: vprp aspect is 16:9.
Writing header...
ODML: vprp aspect is 16:9.
Pos: 16.8s 469f ( 0%) 7.93fps Trem: 838min 11200mb A-V:0.000 [6575:0]

Any tips? Conversion just stopped after about 30s.

Enable it at compilation.
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
Selected video codec: [ffvc1] vfm: ffmpeg (FFmpeg WVC1)
==========================================================================
Movie-Aspect is 1.78:1 - prescaling to correct movie aspect.
x264 [warning]: NAL HRD parameters require VBV max bitrate and buffer size to be specified
x264 [info]: using SAR=1/1
x264 [warning]: DPB size (5 frames, 15667200 bytes) > level limit (4 frames, 12582912 bytes)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast FastShuffle SSEMisalign LZCNT
x264 [info]: profile High, level 4.1
Pos: 0.0s 1f ( 0%) 0.00fps Trem: 0min 0mb A-V:0.000 [0:0]
1 duplicate frame(s)!
Writing header...9f ( 0%) 0.00fps Trem: 0min 0mb A-V:0.000 [0:0]
ODML: vprp aspect is 16:9.
Writing header...
ODML: vprp aspect is 16:9.
Pos: 21.3s 540f ( 0%) 8.70fps Trem: 811min 15164mb A-V:0.000 [7588:0]]
Too many audio packets in the buffer: (4096 in 10186116 bytes).
Maybe you are playing a non-interleaved stream/file or the codec failed?
For AVI files, try to force non-interleaved mode with the -ni option.
Pos: 21.4s 541f ( 0%) 8.72fps Trem: 811min 15145mb A-V:0.000 [7574:0]
Too many audio packets in the buffer: (4096 in 10186116 bytes).
Maybe you are playing a non-interleaved stream/file or the codec failed?
For AVI files, try to force non-interleaved mode with the -ni option.

Flushing video frames.
Writing index...
Writing header...
ODML: vprp aspect is 16:9.

Video stream: 7240.165 kbit/s (905020 B/s) size: 20421072 bytes 22.564 secs 541 frames
x264 [info]: frame I:3 Avg QP:10.49 size: 64271 PSNR Mean Y:72.84 U:85.74 V:85.57 Avg:73.42 Global:60.86
x264 [info]: frame P:315 Avg QP:13.35 size: 31965 PSNR Mean Y:70.60 U:82.15 V:84.17 Avg:71.03 Global:59.38
x264 [info]: frame B:223 Avg QP:13.38 size: 45558 PSNR Mean Y:57.79 U:77.69 V:80.32 Avg:58.75 Global:57.22
x264 [info]: consecutive B-frames: 30.7% 44.6% 6.1% 5.2% 3.7% 4.5% 5.2%
x264 [info]: mb I I16..4: 85.2% 9.3% 5.5%
x264 [info]: mb P I16..4: 6.0% 9.4% 0.5% P16..4: 8.8% 2.0% 1.4% 0.0% 0.0% skip:71.9%
x264 [info]: mb B I16..4: 2.5% 7.4% 0.3% B16..8: 10.6% 0.5% 0.9% direct: 6.5% skip:71.2% L0:56.9% L1:37.1% BI: 6.0%
x264 [info]: 8x8 transform intra:61.0% inter:63.1%
x264 [info]: direct mvs spatial:89.2% temporal:10.8%
x264 [info]: coded y,uvDC,uvAC intra: 61.9% 79.2% 73.9% inter: 9.0% 11.5% 9.8%
x264 [info]: i16 v,h,dc,p: 55% 14% 15% 16%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 10% 13% 49% 4% 4% 4% 5% 4% 7%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 11% 13% 32% 7% 8% 6% 7% 6% 9%
x264 [info]: Weighted P-Frames: Y:23.8%
x264 [info]: ref P L0: 43.3% 19.7% 7.7% 11.0% 8.9% 8.3% 1.1%
x264 [info]: ref B L0: 50.0% 19.0% 19.2% 11.8%
x264 [info]: SSIM Mean Y:0.9988253
x264 [info]: PSNR Mean Y:65.330 U:80.331 V:82.595 Avg:65.983 Global:58.363 kb/s:7240.17

-> Converting AVI file to the MKV container, please wait...
-> Done

-> AVI file size: 19.491 MiB (/storage/public/Random/Fearless.avi)
-> MKV file size: 19.485 MiB (/storage/public/Random/Fearless.mkv)

[microchip8]

For NAL-HRD, you need to use a preset which has vbv-bufsize and vbv-maxrate set. The blu-ray/avchd presets have these. Unless you really need HRD signaling, you shouldn't use it otherwise. Also, the HRD patches are still (very) experimental

For the audio issue, it's a known problem in the default demuxer for TS files. Configure as usual, save all to the batch file, open it up and add to the mencoder line -demuxer lavf then run your encode from the batchfile. See if this fixes it for you

[kripz]

Ok, ill try that next time.

I'm now having another problem with another movie, not sure what it is. I didnt save the mencoder options but here they are from within the job folder.

nice -n 10 /usr/local/bin/mencoder "/storage/public/Random/BOURNE_SUPREM_G51/BDMV/STREAM/00009.m2ts" -o /dev/null -passlogfile "/home/user/.h264enc/job4653/The Bourne Supremacy.log"   -vid 4113      -vf softskip,harddup    -aid 4358 -channels 6 -oac copy 		-ovc x264 -x264encopts pass=1:bitrate=10000:turbo=2:me=tesa:me_range=64:nodct_decimate:nointerlaced:8x8dct:nofast_pskip:trellis=2:partitions=all:mixed_refs:keyint=240:keyint_min=24:psy_rd=0.8,0.2:frameref=10:bframes=7:b_adapt=2:b_pyramid=none:weight_b:weightp=2:direct_pred=auto:subq=10:mbtree:rc_lookahead=60:chroma_me:cabac:aq_mode=1:deblock:level_idc=41:threads=auto:ssim:psnr 
nice -n 10 /usr/local/bin/mencoder "/storage/public/Random/BOURNE_SUPREM_G51/BDMV/STREAM/00009.m2ts" -o "/storage/public/Random/The Bourne Supremacy.avi" -passlogfile "/home/user/.h264enc/job4653/The Bourne Supremacy.log"   -vid 4113      -vf softskip,harddup    -aid 4358  -channels 6 -oac copy 		-ovc x264 -x264encopts pass=2:bitrate=10000:me=tesa:me_range=64:nodct_decimate:nointerlaced:8x8dct:nofast_pskip:trellis=2:partitions=all:mixed_refs:keyint=240:keyint_min=24:psy_rd=0.8,0.2:frameref=10:bframes=7:b_adapt=2:b_pyramid=none:weight_b:weightp=2:direct_pred=auto:subq=10:mbtree:rc_lookahead=60:chroma_me:cabac:aq_mode=1:deblock:level_idc=41:threads=auto:ssim:psnr

Starting the second pass of the encoding process in: 5 4 3 2 1
MEncoder SVN-r29934-4.3.2 (C) 2000-2009 MPlayer Team
success: format: 0  data: 0x0 - 0xf7a51000
TS file format detected.
VIDEO VC1(pid=4113) AUDIO A52(pid=4358) NO SUBS (yet)!  PROGRAM N. 1
Searching for VC1 sequence header... found
VIDEO:  VC-1  1920x1080, 23.976 fps, header len: 30
[V] filefmt:29  fourcc:0x31435657  size:1920x1080  fps:23.976  ftime:=0.0417
==========================================================================
Opening audio decoder: [liba52] AC3 decoding with liba52
Using SSE optimized IMDCT transform
Unimplemented resampler for mode 0xA -> 6 channels conversion - Contact MPlayer developers!
Unimplemented resampler for mode 0xA -> 5 channels conversion - Contact MPlayer developers!
Unimplemented resampler for mode 0xA -> 4 channels conversion - Contact MPlayer developers!
Unimplemented resampler for mode 0xA -> 3 channels conversion - Contact MPlayer developers!
Using MMX optimized resampler
AUDIO: 48000 Hz, 2 ch, s16le, 192.0 kbit/12.50% (ratio: 24000->192000)
Selected audio codec: [a52] afm: liba52 (AC3-liba52)
==========================================================================
Opening video filter: [expand osd=1]
Expand: -1 x -1, -1 ; -1, osd: 1, aspect: 0.000000, round: 0
Opening video filter: [harddup]
Opening video filter: [softskip]
==========================================================================
Requested video codec family [wmvvc1dmo] (vfm=dmo) not available.
Enable it at compilation.
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
Selected video codec: [ffvc1] vfm: ffmpeg (FFmpeg WVC1)
==========================================================================
audiocodec: framecopy (format=2000 chans=2 rate=48000 bits=16 B/s=24000 sample-1)
Movie-Aspect is 1.78:1 - prescaling to correct movie aspect.
x264 [warning]: NAL HRD parameters require VBV max bitrate and buffer size to be specified
x264 [info]: using SAR=1/1
x264 [warning]: DPB size (10 frames, 31334400 bytes) > level limit (4 frames, 12582912 bytes)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast FastShuffle SSEMisalign LZCNT
x264 [info]: profile High, level 4.1
Pos:   0.0s      1f ( 0%)  0.15fps Trem:   0min   0mb  A-V:0.000 [0:0]
1 duplicate frame(s)!
Writing header...3f ( 0%)  4.69fps Trem:   0min   0mb  A-V:0.027 [0:0]
ODML: vprp aspect is 16:9.
Setting audio delay to 0.042s.
Writing header...
ODML: vprp aspect is 16:9.
Setting audio delay to 0.042s.
Pos: 112.2s   2722f ( 1%)  0.21fps Trem:13956min 6193mb  A-V:0.036 [6746:192]

I've never seen it drop down to those speeds before. This was after compiling and updating most of the required programs although its a new Bluray that i got.

Also, instead of defaulting the jobs to the home directory, can you please make it specifiable? my home directory isnt very big.

[microchip8]

did you compile x264 with asm? yasm enabled?

for the job directory, you can already do that in the config file. Look at the TEMPDIR variable in /home/username/.h264enc/config and set it there

[microchip8]

Just came back from the pub
Looking again at your settings, no wonder you'll get such slow speeds (even on Quad cores). You got 10 frame references, 7 bframes with badapt=2, subq of 10, trellis of 2, me_range of 64, weigthp=2, mbtree on, me=tesa and lookahead of 60. This combined with a resolution of 1920x1080 will be pretty slow, even on quad cores and some of the settings are placebo-like (me=tesa f.ex). I bet you're using the insane high quality preset, huh? This preset is a "troll" preset, for those who think they'll see the difference between it and the EHQ preset, and of course for the Japs who love crazy settings

My advice: do not go above the EHQ preset. I always use VHQ but only because my system is a bit slow. Nevertheless, it produces very nice results