h264enc for Linux

[nm]

I think I recall from the AVC forum that PS3 plays MPEG-TS files with H.264 video and AC3 sound.

[blaz]

I think I recall from the AVC forum that PS3 plays MPEG-TS files with H.264 video and AC3 sound.

Hmmm, thanks, I'll have to try this, especially, if we can't get beyond the 2gb limit for avi.

I did ask on the PS3 forum, and was told that the PS3 definitely decodes 5.1 aac, you just need hdmi to your amp to get it.

[microchip8]

Hmmm, thanks, I'll have to try this, especially, if we can't get beyond the 2gb limit for avi.

I did ask on the PS3 forum, and was told that the PS3 definitely decodes 5.1 aac, you just need hdmi to your amp to get it.

well if this is true, then I will enable multi-channel AAC audio for the PS3 presets so if some people have hdmi they can use the 5.1 sound :)

[buzzqw]

I did ask on the PS3 forum, and was told that the PS3 definitely decodes 5.1 aac, you just need hdmi to your amp to get it.

you can set ps2 to put aac 5.1 direct to your amp BUT your amp need aac support
IF your amp don't support aac, you ps3 you can set to decode to STEREO PCM

BHH

PS: i have a ps3, pal 60gb, fw 2.30.. and my amp cannot decode aac :(

[microchip8]

you can set ps2 to put aac 5.1 direct to your amp BUT your amp need aac support
IF your amp don't support aac, you ps3 you can set to decode to STEREO PCM

BHH

PS: i have a ps3, pal 60gb, fw 2.30.. and my amp cannot decode aac :(

good to know buzzqw, thanks for the info :)

[microchip8]

ChangeLog for version 8.4.1
* Updated the keyframe auto detection
* Updated the audio_channels_func() function
* Updated the --help section
* Set the field dominance of the yadif deinterlacer to auto-detection. If the decoder
doesn't export the field order, it defaults to TFF (top field first)
* Enabled multi-channel AAC audio for the Sony PS3 presets
* Small update to the FPS conversion menu
* Fixed a typo in the color() function
* Small menu enhancements to the video_interlace_func() function
* Added support for video bitrate calculation when using 5 channels PCM audio
* Removed delay when exiting from user interrupt
* Updated the audio codecs menu
* Updated the README.encoding file

[microchip8]

ChangeLog for version 8.4.2
* Fixed a problem where the detected resolution based on the
crop values favored the values of the cropdetect filter instead of the
user provided values. Reported by blaz from the Doom9 forum
* Renamed resolution variable 'res' to 'vres'
* Added support for doing 25 -> 50 fps and 29.970 -> 59.940 fps conversion using simple frame duplication
* Updated the man page

[nfm]

Much appreciated work froggy1! Thank you very much for great knowledge, skill, and time.

I'm looking to convert DVD into H264 using x264. It's been a while since I've done it in GNU/Linux. I remember I used to pipe ffmpeg with x264 (source -> ffmpeg -> y4m | x264 mypreset). This way I was able to use preset from megui (hdslow) instead of confusing mencoder options. Does mencoder these days features all x264 features?

[nm]

I'm looking to convert DVD into H264 using x264. It's been a while since I've done it in GNU/Linux. I remember I used to pipe ffmpeg with x264 (source -> ffmpeg -> y4m | x264 mypreset). This way I was able to use preset from megui (hdslow) instead of confusing mencoder options. Does mencoder these days features all x264 features?
MEncoder calls libx264's parameter parsing code, so all parameters are always supported and the syntax is basically the same as in x264. Only x264 features that are not supported are the GTK+ GUI and Matroska and MP4 output since that part is handled by MEncoder. However, MP4 output should work with libavformat's muxer (-of lavf -lavfopts format=mp4).

[microchip8]

ChangeLog for version 8.4.3
* Clarify that the aspec ratio is the one for input sources
* Added MP4Box -ipod switch for the iPod/iPhone profiles
* Set the field order of the frame doubling yadif deinterlacer to auto detection
* Implemented source_fps_func() function to reduce code duplication. This function
automatically detects the FPS of the content and is shared between the deinterlace
and interlace functions
* Added support for applying denoising/deblocking on interlaced content. This is done
by deinterleaving the content with the il=d filter, applying the denoising/deblocking on
the deinterleaved content and interleaving it back with the il=i filter. That way we don't
break the interlacing
* Color enhancements to the -bpp and -bpb options

[rberger]

MEncoder calls libx264's parameter parsing code, so all parameters are always supported and the syntax is basically the same as in x264.

It even seems to be *exactly* the same up to a few parameters where mencoder has it's own version or plain wouldn't allow the x286 parameter because it could interfere with mencoders own operation, like -passfile for --stats or --progress leading to premature exit since it would mess with mencoders output.

Apart from those few, x264 parameters seem to be handed down by mencoder untouched. The reason they may have different names in the mencoder docs is because some options are implemented with several names in x264 itself, like --filter being the same as --deblock. Apparently, you can always look into common/common.c in the x264 source tree to find out what is what.

Only x264 features that are not supported are the GTK+ GUI and Matroska and MP4 output since that part is handled by MEncoder.

Seems to me all those were never part of x264 proper. It's just an encoder library with a CLI.

However, MP4 output should work with libavformat's muxer (-of lavf -lavfopts format=mp4).

I wouldn't recommend that. As of SVN-r26492, mencoder's MP4 muxing is once again - and likely always has been - broken. It even warns about that.

There's always the possibility to encode video and audio separately with mencoder and mux with gpac/mp4box. This generally works pretty well with PAL sources, but the moment you do IVTC on NTSC material or other frame juggling filtering, you might run into A/V sync problems. In these cases, it really matters what options you feed to mencoder, and it becomes a kind of a trial and error situation.

See e.g. sections 14.1.9 and 14.1.12.1 here:
http://www.mplayerhq.hu/DOCS/HTML/en/menc-feat-dvd-mpeg4.html

and how people already have run into sync troubles
http://forum.doom9.org/showthread.php?t=122018
http://thread.gmane.org/gmane.comp.video.mplayer.user/48954
http://thread.gmane.org/gmane.comp.video.mplayer.user/49206

That's one of the reasons encoding on Linux still sucks, that while mencoder seems to be the most advanced full feature transcoding frontend (transcode being another) and often enough working quite well right out of the box, it might always unexpectedly fail in some only slightly subtle situation. MP4 muxing definitely seems to be one of those.

[nm]

Seems to me all those were never part of x264 proper. It's just an encoder library with a CLI.
Well, they are part of x264, but not the core library (libx264). GUI and MP4 support need to be enabled when configuring the build (and they need external libraries, of course). Matroska output support is built-in to the CLI.

I wouldn't recommend that. As of SVN-r26492, mencoder's MP4 muxing is once again - and likely always has been - broken. It even warns about that.
Ok, I was actually thinking whether I should say that it might work or if it should work, and I opted for the latter since there were rumors that it now works. I wouldn't recommend using it either if there is doubt.

That's one of the reasons encoding on Linux still sucks, that while mencoder seems to be the most advanced full feature transcoding frontend (transcode being another) and often enough working quite well right out of the box, it might always unexpectedly fail in some only slightly subtle situation. MP4 muxing definitely seems to be one of those.
Avidemux is also worth trying, although it's more GUI-oriented. At least it can output MP4 properly :)

[rberger]

Avidemux is also worth trying, although it's more GUI-oriented. At least it can output MP4 properly :)

Right, thanks for the reminder. I looked at avidemux at one point during the past weeks, but wasn't that much impressed, don't ask me why.

Also, I silently assumed it would link against ffmpeg libraries anyways and hence offer no real functional advantage compared to mplayer. Doesn't seem to be the case though, albeit I'm not yet completely sure about that. If they really maintain own muxers working better than the ffmpeg variants, that would definitely be great (although I have a distinct feeling that linking against gpac would be the best thing all these projects could do, as it appears to be the most advanced and robust mp4 muxer).

Considering avidemux offers a CLI, which escaped me the other day, and they advertise scriptability, I'm almost ready to get a little excited.

If anybody shared their avidemux experiences, especially regarding AVC/AAC coding and MP4 muxing, I would sure appreciate it.

[nfm]

With ihq (Insane High Quality) preset mencoder crashes, uhq preset works fine. @nf and @rberger, thanks for good info, I dumped mp4 in favor of mkv in linux, although I still choose mp4 in windows.

Edit: Looks like mencoder crashes with custom x264 settings too:
#!/bin/bash
#

mencoder "/media/vol05/DVD Rips/MainMovie/INSIDE MAN/VIDEO_TS/VTS_01_2.VOB" -o /dev/null -fps 30000/1001 -ofps 24000/1001 -frames 1000 -vf pullup,crop=720:352:0:64,softskip,harddup -af volnorm=2 -oac faac -faacopts mpeg=4:br=110:object=2 -ovc x264 -x264encopts pass=1:bitrate=1000:ref=5:mixed-refs:no-fast-pskip:bframes=5:b-pyramid:b-rdo:bime:weightb:direct=auto:filter=-2,-1:subme=6:trellis=2:partitions=p8x8,b8x8,i4x4,i8x8:8x8dct:me=umh:threads=auto:thread-input:progress:no-dct-decimate:no-psnr:no-ssim -passlogfile ./h264.stats 2>/dev/null

mencoder "/media/vol05/DVD Rips/MainMovie/INSIDE MAN/VIDEO_TS/VTS_01_2.VOB" -o ./ouput.avi -fps 30000/1001 -ofps 24000/1001 -frames 1000 -vf pullup,crop=720:352:0:64,softskip,harddup -af volnorm=2 -oac faac -faacopts mpeg=4:br=110:object=2 -ovc x264 -x264encopts pass=2:bitrate=1000:ref=5:mixed-refs:no-fast-pskip:bframes=5:b-pyramid:b-rdo:bime:weightb:direct=auto:filter=-2,-1:subme=6:trellis=2:partitions=p8x8,b8x8,i4x4,i8x8:8x8dct:me=umh:threads=auto:thread-input:progress:no-dct-decimate:no-psnr:no-ssim -passlogfile ./h264.stats 2>/dev/null
Screw mencoder, looks like I have do try the old way, ffmpeg | x264.

[rberger]

Throw out 'thread-input' and 'progress', then try again (I mentioned the issue with progress above already).

[nm]

Also, I silently assumed it would link against ffmpeg libraries anyways and hence offer no real functional advantage compared to mplayer. Doesn't seem to be the case though, albeit I'm not yet completely sure about that. If they really maintain own muxers working better than the ffmpeg variants, that would definitely be great
They use the lavf MP4 muxer, which works just fine AFAIK. The problem seems to be in MEncoder and its b-frame handling.

(although I have a distinct feeling that linking against gpac would be the best thing all these projects could do, as it appears to be the most advanced and robust mp4 muxer).
I doubt it would improve the situation in this case. GPAC is also a bitch to compile.

If anybody shared their avidemux experiences, especially regarding AVC/AAC coding and MP4 muxing, I would sure appreciate it.
Never had a problem with it for these purposes and this seems to be what most people are now using it for. Personally I prefer MEncoder, but that's because I already know the tricks to make it work.

[nm]

Edit: Looks like mencoder crashes with custom x264 settings too:
In addition to rberger's solution, there are a few other things to change:
When trying new stuff, don't redirect error messages to /dev/null. That is, remove "2>/dev/null" from the end of the command.
Now you missed the helpful message "Option x264encopts: Unknown suboption progress" because of this.
Use turbo=1 or turbo=2 for the first pass (or just use CRF instead of bitrate modes)
Only use AVI output for H.264 if that's the last option. Try encoding audio and video separately (use -of rawvideo for H.264 ES output) and mux the resulting streams with mkvmerge or MP4Box.
Screw mencoder, looks like I have do try the old way, ffmpeg | x264.
How do you intend to do IVTC in that workflow? I'd replace ffmpeg with MEncoder for better filtering, even if you insist on calling x264 separately.

[rberger]

They use the lavf MP4 muxer, which works just fine AFAIK. The problem seems to be in MEncoder and its b-frame handling.

Which means the same muxer as mencoder. And the latter failed on me with the only x264 option specified being crf, which means zero b frames. Maybe it's mencoder, but frankly it'd surprise me.


Never had a problem with it for these purposes and this seems to be what most people are now using it for. Personally I prefer MEncoder, but that's because I already know the tricks to make it work.

Thanks for the feedback. In the meantime, I checked again and found once more that avidemux is not really what I want, effectively being too gui bound. The quest continues.

[nfm]

I got it to work, taking out 'thread-input' and 'progress' did the trick. Pardon me but I'm very much newbie in encodings, always will be :p
Regrading the "2>/dev/null", that' why I didn't see errors, now I understand that the standard output was redirected to /dev/null.
Use turbo=1 or turbo=2 for the first pass (or just use CRF instead of bitrate modes) Will do nm, thanks.
I tried matroska container "-o output.mkv -of lavf" there's no audio when faac is used, there's a problem with audio resampling rate. I still haven't got gpac to compile. I'm no doom9 guru, but isn't there a problem storing b-frames within an avi? Is mencoder affected by this? I think I will go about encoding audio and video separately, and mux them into mkv. Again, nm and rberger, big thanks.

[rberger]

Don't apologize, I'm pretty much new to the subject myself, and we all have to start somewhere and somehow.

Regarding muxing into Matroska (mkv), I believe its not supported by mencoder. It only has a demuxer (i.e. it can read) for that container type, but not a muxer (can not write). If it had one, your command line would anyway look like this "-o output.mkv -of lavf -lavfopts format=mkv". But as said I'm pretty much sure that's not supported.

Look at the output of your mencoder run. At some point early on it likely tells you that it muxes into avi (the default output container format) since your options make no sense to mencoder. That's also likely the cause your sound isn't muxed in, because avi doesn't support AAC.

Regarding b frames and avi I don't know, being not that proficient with this format myself.

[nm]

Which means the same muxer as mencoder. And the latter failed on me with the only x264 option specified being crf, which means zero b frames. Maybe it's mencoder, but frankly it'd surprise me.
Yes, MEncoder uses the same libavformat MP4 muxer, but it does work fine when used through ffmpeg and Avidemux. People use it all the time.

Thanks for the feedback. In the meantime, I checked again and found once more that avidemux is not really what I want, effectively being too gui bound. The quest continues.
Well, the only scriptable tool that is better than MEncoder for handling telecine and variable framerate scenarios is AviSynth (since its IVTC filters support writing timecode files). I'd suggest using it through Wine if you have an x86 system.

[nm]

I'm no doom9 guru, but isn't there a problem storing b-frames within an avi? Is mencoder affected by this?
Yes, there are problems. If the audio is AVI-compliant, it can be muxed with H.264 video but the A/V sync may be off a little. This depends on how many B-frames the encoder uses in the first batch that it inserts, IIRC. AVI output may be the last option if you have otherwise problems keeping the sync when encoding hybrid NTSC sources (part 24000/1001 fps film, part 30000/1001 fps video).

[buzzqw]

@froggy1

just a simple request

would be possible to start h264enc (and btw xvidenc/divxenc) with only specify the input file ?

just start asking only the input file, then start asking pass/preset/audio/fps.. and so on
would be a little easier

one more: the script can check if a new version is released and download id

thanks

BHH

[microchip8]

@froggy1

just a simple request

would be possible to start h264enc (and btw xvidenc/divxenc) with only specify the input file ?

just start asking only the input file, then start asking pass/preset/audio/fps.. and so on
would be a little easier

one more: the script can check if a new version is released and download id

thanks

BHH

unfortunately this is not possible at the moment as i'll have to rewrite a lot of stuff and i'm a bit lazy at the moment to do that... as for checking for a newer version, i think i can add this

[buzzqw]

thanks for your answer, i will glady wait :P

BHH

[Lokean]

I looked at avidemux at one point during the past weeks,

Also, I silently assumed it would link against ffmpeg libraries anyways and hence offer no real functional advantage compared to mplayer. Doesn't seem to be the case though, albeit I'm not yet completely sure about that.
Avidemux uses an internal copy of the ffmpeg libraries (i.e. it doesn't link to the system ffmpeg).

[blaz]

froggy1's programs use dd or pv to create iso's from a dvd.

I've never heard of pv and am running into walls searching for it.

Can someone (froggy1?) point me to it (on the web ?) and maybe provide any advice as to whether it is preferable to dd.

One problem I'm having is copying older scratched dvds.

[microchip8]

well, i don't know which distro you use, but on openSUSE, pv is in the Packman repository. A search on rpmfind, finds this utility http://rpmfind.net/linux/rpm2html/search.php?query=pv

pv is an alternative to dd and displays a nice progress bar when making an ISO image out of a DVD. As for scratched DVDs, there's little I can do about it, it's beyond my control

[blaz]

Hmmm, I had looked at the pv package on debian but somehow: "PV ("Pipe Viewer") is a tool for monitoring the progress of data through a pipeline" didn't seem like what I was looking for.
But now on installing it and reading the man page, I see that it does a lot more than that.
Will give it a try. Thanks again.

[microchip8]

ChangeLog for version 8.4.4
* Small menu enhancements to the FPS settings
* Fixed wrong explanation for the temporal denoiser
* Removed support for the Sonic and SonicLS audio codecs since they
are experimental and far from complete for daily use
* Updated the man page

[microchip8]

ChangeLog for version 8.4.5
* Fix for the sdb, sdm, sdh, hdb, hdm and hdh presets. These presets are only
supported in 1-, 2-, or 3-pass mode since they use VBV maxrate and VBV bufsize
with is not compatible with QP/CRF encoding modes

[microchip8]

ChangeLog for version 8.4.6
* Merged checking of the encoding modes for the specific presets that need it
* Removed the Black & White presets since they are useless and the only difference
in them from the others is that they don't use chroma motion estimation
* Updated the man page

[palpyhat]

Does the script support selecting SAR or PAR to be input directly into the MPEG-4 video stream (not the container)?

The same question also applies to xvidenc.

[microchip8]

not at the moment.

[microchip8]

ChangeLog for version 8.4.7
* Added support for inclusion of external subtitles when encoding video files.
The external subtitle will be hard coded into the encode
* Updated the man page

[WalterK]

Thanks for the update info on all three, time for an update of all three been putting it off for months now.

[microchip8]

ChangeLog for version 8.4.8
* Disabled the use of unique temporary directories to store temporary
configuration files. This fixes an MP4Box crash on newer systems

[microchip8]

ChangeLog for version 8.4.9
* Bugfix: don't leave config files upon user interrupt
* Removed reference to the Sonic audio codec in the -sc option as this codec is no longer supported

[microchip8]

ChangeLog for version 8.5.0
* Added support for using the neroAacEnc AAC audio encoder with LC, HE and HEv2 profiles
* Function renaming: aacplus_audio_func() -> neroaac_audio_func()
* Variable renaming: aacplusbr -> neroaacbr
* Audio codec renaming: AAC+ -> neroAAC

[microchip8]

ChangeLog for version 8.5.1
* Added support for using the neroAacEnc AAC audio encoder when using the
portable device presets icvp, ichqvp, ivp, ihqvp, ipvp, iphqvp, avp, ahqvp, nks60vp,
nks60hqvp, pspvp, psphqvp, ps3vp, ps3hqvp, zvp, zhqvp, xvp and xhqvp
* Bugfix: when using the f9 or f9hq preset with the neroAAC audio codec, -oac pcm was
not passed to the MEncoder options
* Changed default unsharp mask / gaussian blur matrix to chroma & luma 5x5 as it provides
a better balance than the 3x3 matrix

[microchip8]

ChangeLog for version 8.5.2
* Feature request: add the possibility to specify the path to MEncoder.
Useful if the user has two or more versions of MEncoder but wants to use a
specific one for the encoding
* Check to see if destination directory is writable by writing a small file to it.
* Updated the README.h264enc file
* Updated the man page

[microchip8]

ChangeLog for version 8.5.3
* Feature request: add the possibility to disable MEncoder's A/V sync
algorithm when encoding video files. This can be used to fix input video files with
bad A/V sync in them, however there are no guarantees that this will actually fix the problem
* Added initial support for the TS container. This requires tsMuxeR and MP4Box
(for audio/video dumps). When muxing to the TS container, only the FAAC AAC and
AC3/copy audio codecs are supported. Read the README.h264enc file which provides
more information on where to get tsMuxeR.
* Updated the -sc option, added tsMuxeR to the list
* Enabled back the use of temporary directories based on process ID to store config files
* Fixed a bug in the check_audio_codec_func() function. Variable MENOCDER should be MENCODER
* Updated the requirements part in the README.h264enc file
* Small clean ups
* Updated the man page

[blaz]

In your latest script:
- Not sure how to specify ac3 pass through for eventually muxing into an mpeg ts stream
- I get the following error message:

-> Utility 'tsMuxeR' or 'MP4Box' is missing!
-> Skipping AVI to TS conversion!

But I can envoke both at the command line. For example:
SmartLabs tsMuxeR. Version 1.7.3(b) http://www.smlabs.net
Usage: (and so on)

[microchip8]

this is strange as it works here. Are you sure that tsMuxeR is in your path? eg in /usr/local/bin or similar? the script just checks with the "which" command if tsMuxeR and MP4Box are in your path, nothing else

the TS container is only supported with AAC, AC3 or copy of the AC3 stream. I'm adding support for DTS at the moment

[talen9]

h264enc was reviewed on Linux.com (here (http://www.linux.com/feature/140081)) :)

[microchip8]

h264enc was reviewed on Linux.com (here (http://www.linux.com/feature/140081)) :)

hehehehe, nice. Didn't expect a review from Linux.com :D

[blaz]

OK, thanks. I was prepending "~/bin" to the path and exporting that.
Which works in the bash shell, but not with "which" or your script.
But everything works when I prepend the full path (not "~").

As for copying ac3 audio, this is what h264enc is giving me:

-> Scanning for audio streams...

Audio stream: 0 Format: AC3 (5.1) Language: en ID: 128.
Audio stream: 1 Format: AC3 (stereo) Language: en ID: 129.
Audio stream: 2 Format: AC3 (stereo) Language: fr ID: 130.
Audio stream: 3 Format: AC3 (stereo) Language: en ID: 131.

Specify the DVD Audio Stream ID [default is 128]:
Specify the Audio Codec [AAC/neroAAC - default is AAC]:

When I take the dfault (128) there isn't an option to just copy the ac3 over?
I'm using options: -1p -p ps3hqvp

Thanks ...

[microchip8]

Ah yes, I haven't updated the PS3 profiles yet - try now http://www.mediafire.com/?hyzthemjcj7

[microchip8]

@ blaz
on a further not, are you sure that the PS3 can handle H.264 TS files? All I could find on the Net was that it can play MPEG2 TS and PS files but not H.264 TS ones

[blaz]

@ blaz
on a further not, are you sure that the PS3 can handle H.264 TS files? All I could find on the Net was that it can play MPEG2 TS and PS files but not H.264 TS ones

Yes, but only if i first massage the h264 with h264info.
I've got a thread here about it: http://forum.doom9.org/showthread.php?t=137517

I was hoping that your script would include some missing mencoder options that would eliminate that h264info step.

Haven't got that far yet, but that new version does run with ac3 copy and the ps3 options.

Will not be looking at this for another week (gotta travel now), but will get back to you eventually if it works with the PS3.