avi2mp4.sh: Convert files to MP4 (H264 or XviD/AAC)

[micmac]

avi2mp4.sh README

____ ____ ____ ____ ____ ____ ____ ____ ____ ____
||a |||v |||i |||2 |||m |||p |||4 |||. |||s |||h ||
||__|||__|||__|||__|||__|||__|||__|||__|||__|||__||
|/__\|/__\|/__\|/__\|/__\|/__\|/__\|/__\|/__\|/__\|


-------------- Converts video files in batch mode to MP4 ---------------

Where to get it: http://forum.doom9.org/showthread.php?t=133598

Before putting this script to serious use encode a few samples first.

avi2mp4.sh is a batch converter. For each file a resolution gets calcu-
lated and a compressibility test determines the minimum bit rate needed
for the level of quality the user asks for.

If this isn't possible with the maximum bitrate the user allows, then
the output resolution gets downsized and a new compressibility test
starts.

Once a matching pair of bit rate and resolution has been found the input
file gets converted to H.264/MPEG-4 AVC or MPEG-4 video and AAC audio.
Finally these streams are wrapped into MP4.

Different kinds of subtitles can be added: SRT, SUB, 3GPP (TTXT) and
VobSub.

----------------------------- Requirements -----------------------------

- bc calculator
- faac or neroAacEnc (even Wine and a Windows binary will do)
- mplayer
- mencoder
- aacgain (at least v1.7.0; you can find a Debian package at RareWares:
http://www.rarewares.org/debian/packages/unstable/)
- MP4Box (part of gpac; v0.4.4 or later required)

------------------------- Compressibility Tests ------------------------

The compressibility test is new. Send me an e-mail with concerns or
ideas you might have.

--------------------------------- Hints --------------------------------

Global options need to be set in a file in your user's home directory,
namely ~/.avi2mp4.conf. An initial configuration file will be created.

MEncoder is used for the video conversion. There are multiple possibili-
ties to influence MEncoder's behaviour. The first is the configuration
variable MENCODER_OPTS. The content will be passed on to MEncoder.

A second possibility is to add MEncoder options on a per file basis.
Put the desired options in a file and name it like the input file name
extended by the suffix ".me" (e.g. movie.avi.me).

You can add video filters on a per file basis as well. Write them into
a file the same way as before, only this time use the suffix ".vf". Your
filters will be prepended to the existing chain (scale=x:y,harddup).

avi2mp4.sh searches the video filter chain for "crop" statements. If one
is found the calculation of the output resolution will be adjusted
accordingly.

You can indicate that a video is Progressive NTSC by creating a file
with the suffix ".p" (frame rate of the output video will be 23.976
frames per second for NTSC input).

--------------------------- MPEG containers ----------------------------

MPEG containers are problematic because they don't contain a proper
index like AVI containers do. This wasn't a problem until the
compressibility test was introduced. Since then avi2mp4.sh relies on
being able to "jump" through the input file. More often then not this
didn't work with MPEG containers.

Because of that input files are now skipped when an MPEG container is
detected. You'll have to re-multiplex the files to, e.g., AVI:

mencoder -ovc copy -oac copy input.mpg -o movie.avi

To rip a DVD simply dump the title you want to convert to your disk
first:

mencoder dvd:// -ovc copy -oac copy -o movie.avi

Finally:

avi2mp4.sh movie.avi

-------------------------------- iPods ---------------------------------

You want your videos to play on your iPod?

- obey bit rate and resolution limits
- no "B" frames allowed. Also, with H.264, the baseline profile level
must be set and CABAC disabled
XVID_ENCOPTS=max_bframes=0
X264_ENCOPTS=nocabac:level_idc=30:bframes=0
- AAC's Low Complexity (LC) profile must be used

Videos created with the initial configuration file should work on your
iPod out of the box. Same should be the case for your PSP.

Please also read about the configuration variables "MP4BOX_IPOD" and
"SBTL".

----------------------------- Configuration ----------------------------
...

[Adub]

Cool, thanks for your work!

[rootkit]

I had a similar tool here: http://teknoraver.campuslife.it/software/mp4tools/
but mine can encode any input, not just .avi and with every aspect ratio.
Would you improve mine instead of writing your own?

[micmac]

I had a similar tool here: http://teknoraver.campuslife.it/software/mp4tools/
but mine can encode any input, not just .avi and with every aspect ratio.
Would you improve mine instead of writing your own?

Hi rootkit!

I looked at your script and figured it'd take me some time to get to know it. Sorry.

In the meantime I've changed avi2mp4.sh to support any file and any aspect ratio as well.

1.0.5: - Small change to comments
- Use myidentify instead of midentify; now input pixel aspect
ratio doesn't have to be 1:1 anymore
- Function c; unset variables containing essential file
properties and continue
- Protect mplayer from input while extracting audio
(-nolirc -noconsolecontrols etc.)
- Remove AVI requirement; now any file is possible as input
- Add VFOPTS variable
1.0.6: - Remove '-noskip -mc 0' again
- Change "Usage: ..." info
- Add rough sketch about what the script does and what it can't
do
- Add note about iPod (tm) import
- Print target bitrate and bpp
- Unset LC_ALL and set LC_NUMERIC to POSIX to make sure we get
dot instead of comma as decimal separator as bc expects a dot
1.0.7: - Add a little enhancement to function c; ability to return to
the n-th eclosing loop
- Fix two funtion c calls returning to the wrong loops

Regards
micmac

[micmac]

1.0.8: - Remove myidentify log when skipping file
- Use rm -f
- Add possibility to add options to mencoder calls through
$MENCODER_OPTS

[microchip8]

http://h264enc.sourceforge.net :)

[micmac]

1.0.9. - In case aspect ratio is not set simply assume Width:Height
- Add -endpos .1 to myidentify to prevent playback of audio
files; thanks to Reimar Doeffinger for the tip
- Check $VFOPTS for cropping. If detected calculate new aspect
ratio so we get a proper output resolution

[micmac]

http://h264enc.sourceforge.net :)

Hi!

I can't say that I tried it but it seems h264enc isn't what I was looking for. I didn't want an interactive application. Instead I wanted an application for one specific task with batch processing capabilities. That's why I came up with avi2mp4.sh.

Regards
mic

[micmac]

1.1.0: - Remove bogus comma from sed call (BASENAME="...")
- Remove redundant scale command from bc call
- Check for (and prevent) upscaling of video
- Lower the bitrate with regards to BPP
- Move crop detection around
- Update crop detection for special cases w=0 and h=0; this
also fixes a potential division by zero
- Update comments
- Add CH and CW to function c so both get unset
- Dimension -> Resolution
- Display $CW x $CH as resolution if available
- Make sure ID_VIDEO_WIDTH, ID_VIDEO_HEIGHT, ID_VIDEO_FPS and
ID_AUDIO_NCH aren't negative
- Fix the for loop that looks at output resolutions
1.1.1: - Fix behaviour when supplied cropping values are bogus
1.1.2: - Another attempt to correct the behaviour in case of bogus
crop values; only accept numbers and let mencoder do the
dying
- Change loop expression in c-style for-loop once more just for
correctness' sake

[micmac]

1.1.3: - Direct error messages to stderr
- Minor comment fix
- If there's a ".p" file treat video as progressive
- Users can add mencoder options on a per-file basis
- Add tip about DVD ripping
- Add clear_variables function

[microchip8]

Hi!

I can't say that I tried it but it seems h264enc isn't what I was looking for. I didn't want an interactive application. Instead I wanted an application for one specific task with batch processing capabilities. That's why I came up with avi2mp4.sh.

Regards
mic

h264enc has iPod presets, batch encoding support and support for MKV/MP4 and OGM :)

[micmac]

h264enc has iPod presets, batch encoding support and support for MKV/MP4 and OGM :)

I like my script :)

[microchip8]

:) of course,.... I was just pointing out to your previous reply on h264enc

[microchip8]

you should really add the 'harddup' filter at the end of the filter chain so duplicate frames get hard-coded in the file. This is essential if you're going to remux the AVI to another container so you won't lose A/V sync

also -mc 0 completely disables the A/V sync code of MEncoder so you should really stay away from it

[micmac]

Hi!

you should really add the 'harddup' filter at the end of the filter chain so duplicate frames get hard-coded in the file. This is essential if you're going to remux the AVI to another container so you won't lose A/V sync
harddup has been used since the very first version:
mencoder ... -vf ${VFOPTS}scale=${OUT_HORIZ}:${OUT_VERT},harddup ...

also -mc 0 completely disables the A/V sync code of MEncoder so you should really stay away from it
That's what I'm thinking as well ("-mc 0 -noskip" were removed from the script in v1.0.6).

Regards
mic

[microchip8]

ahhh must have missed harddup then, sorry bout that ;)

[micmac]

Thanks for all the downloads! If you're missing a feature or if there's something else, let me know.

Btw to get around the "long filename crashes" with gpac/MP4Box, recompile it _with_ CFLAGS but without any "-OX" options.

E.g.

CFLAGS="-march=athlon-xp -pipe"
is fine, whereas
CFLAGS="-march=athlon-xp -pipe -O2"
is not. This workaround helped in my case.

Regards!

[micmac]

1.1.4: - Add possiblity to set maximum output resolution, thanks to
L. Pearce for the idea; see comments regarding $MAX_RES
- Changed comments regarding MP4Box crashes (added suggestion
for real workaround)

[micmac]

1.1.5: - Add $MP4BOX_IPOD to set whether "MP4Box -ipod ..." is run
- Add possibility to renice the script (alter its priority)
- use "-oac pcm" instead of "-oac copy"; it does add a little
overhead (and we need more drive space) but it helps keeping
audio and video in sync; also some audio codecs cannot be
muxed into AVI, e.g. Vorbis, so now we're working around this
issue

[micmac]

1.1.6:
- Split release into ChangeLog, README, INSTALL and script itself
- Add possibility to use neroAacEnc AAC encoder instead of faac; it's
not Open Source but I ran into an issue with faac earlier and
apparently it's not maintained anymore
- Increase accuracy of aspect calculation; I doubt this has any effect

[micmac]

1.1.7:
- mention faac in INSTALL
- add possibility to override neroAacEnc binary (README for more
info)
- add Xvid support; it's so much faster than x264, had to add it
- add configuration file example to README

[micmac]

1.1.8:
- use random ASCII file names to work around problems with special
characters (e.g. with MPlayer's pcm writer and gpac)
- make the script a little bolder ;)

[micmac]

1.1.9:
- make sure muxing doesn't result in variable video frame rate
- cleaned up quite a bit (quoting, brackets, ...)
- better error detection
- clean user configuration file before sourcing it
- write initial configuration file
- got rid of all evals
- removed hardcoding of iPod-specific encoder options

[micmac]

1.1.10:
- remove BPP usage
- add compressibility check
- delete INSTALL file (put content into README)
- clean up README
- (hopefully) last batch of quoting fixes
- make scripting style more consistent
- better sed scripts
- use $HOME instead of "~" expansion
- move command substitutions out of [tests]

[micmac]

1.1.11:
- during compressibility test capture mencoder output in a file
and display content of file in case mencoder exits with an
error, otherwise user wouldn't know what went wrong
- used long command options when available
- don't write/source temporary configuration file
- don't just source the mplayer -identify output either for that
matter
- only load necessary variables from configuration file and
mplayer's identify output
- use random name for myidentify output, too
- set sane PATH, IFS and umask
- clear all aliases and the command path hash
- prevent core dumps
- end shebang line with hyphen
- changed name of array media_vars to all_media_vars and added
an array called mplayer_media_vars for clarity's sake
- made parsing of file for variable a function
- changed the initial configuration a bit
- refuse to crop repeatedly
- demand Bash, version 2 or greater

[micmac]

1.1.12:
- added subtitle support (SRT, SUB and VobSub)
- added $TRACK_LANG so user can set a three digit language code.
It's not _that_ useful because you can not set it on a
per-file or even-per track basis. Why did I even add it in the
first place? :D
- call MP4Box from within the tmp directory do avoid another bug
(for some reason MP4Box added something like ".idx" to the
command line otherwise)
- some small fixes (comments, README)
- call MP4Box without "-isma" because it removes the subtitles

[micmac]

1.1.13:
- script now refuses to (try to) convert mpeg containers
- updated README

[micmac]

1.1.14:
- now 3GPP text subtitles (.ttxt) are recognized as well
- new configuration variable SBTL, see README
- pass TRACK_LANG & video fps along as a global MP4Box argument
- changed subtitle handling, now for text subtitles the 3GPP
header is adapted slightly for proper display (I tried to
follow the findings of this blog entry:
http://blog.jharding.org/2008/11/subtitles-on-appletv-and-iphone.html
Thanks for the ideas!)
- use "-ipod" argument directly when muxing

[micmac]

1.1.15:
- fix subtitle related mistake
- set vim modeline
- some cleanups (in README as well)

[micmac]

1.1.16:
- fix quite a few return checks that caused avi2mp4.sh to not terminate
properly
- adjust ChangeLog's style :)

[microchip8]

Added to my openSUSE build service repo where I offer packages for Fedora, openSUSE and SLES/SLED

http://download.opensuse.org/repositories/home:/microchip8/

[micmac]

Thanks! :)

[micmac]

1.1.17:
- Fixed a few spelling error
- Changed initial configuration (point out a few differences between PSPs and
iPods

[micmac]

1.1.18:
- Fixed the if-block that deals with cropping and updating the aspect ratio

[micmac]

1.1.19:
- Use wildcard when cleaning up instead of trying to delete file by file
- Changed default config for PSPs/iPods, apparently x264 changed b frame
behaviour or default config regarding b frames
- Substitute some more short command options with long ones
(wc/bc/grep/renice)
- Recomplicated track language handling again to fix VobSub track languages
(these are now determined by MP4Box as it should be); the video track does
not get a language assigned anymore as that was more or less stupid anyway;
the language code is now applied only to audio and text subs