M2TS, FFMPEG, Sony Vegas, and Me

[TheLostSkeleton]

I wasn't 100% sure where this should go, since it could technically fall under video capture, alternative A/V containers, MPEG4, and maybe Virtual Dub. So maybe I'm a big idiot, but I've been tearing my hair out about this for a week now; at the very least maybe somebody here could point me to another forum where I could get the answers I need. Regardless, this is going to be kind of a long post, so buckle in.

Anyway!

I bought a Roxio Gamecap HD Pro because it was the cheapest high-def capture device and it seems you get what you pay for.

The device records to Blu-ray style M2TS files. The primary issue is that I cannot edit these M2TS files in Sony Vegas; Vegas does not understand the flavor of AAC it is using. I have thrown these M2TS files at everything I can think of; Avidemux, tsmuxer, easyh264, VirtualDub, etc. Sometimes it will convert the files but take an impractical amount of time (4+ hours for a 15 minute clip), other times it will convert files quickly but Vegas still has trouble. Roxio's official solution is to use their sluggish, bloated Videowave software to convert to MP4, but that gives me no control over bitrate and produces sub-par results.

I decided to turn to FFMPEG, because FFMPEG can handle anything. Unfortunately, either FFMPEG cannot handle these files, or I am using FFMPEG wrong.


Attempted Solution #1

ffmpeg.exe -i %1 -c:v mjpeg -b:v 60000k -c:a pcm_s16le %1_COPY.avi

Simple. Convert the video to the two most basic codecs imaginable. Short test encodes imported in to Sony Vegas just fine, full-video encodes create a curious problem where Vegas sees the audio track, but only thinks it is three seconds long. As it turns out, if FFMPEG has to convert one of these M2TS files, any video over 2 minutes and 46 seconds long generates this shortened audio problem. I have no idea why. The files themselves play back fine in Media Player Classic, the issue is just in Sony Vegas.


Attempted Solution #2

ffmpeg.exe -ss 00:02:46.000 -i %1 -to 00:02:44.000 -c:v mjpeg -b:v 60000k -c:a pcm_s16le %1_COPY.avi

With the above in mind, maybe you could just seek past the problem and just make a bunch of 2:45 clips and stitch them together after the fact, right? Nope. As far as I can tell, anything past the 2:46 mark has this problem.


Attempted Solution #3

ffmpeg.exe -i %1 -c:a pcm_s16le %1_COPY.wav
ffmpeg.exe -i %1 -i %1_COPY.wav -shortest -map 0:0 -map 1:0 -c:v mjpeg -b:v 60000k %1_COPY.avi

Extract the audio, convert it to PCM, re-add it to the video. Generates the 3-second-long-audio-track issue in Sony Vegas.


Attempted Solution #4

ffmpeg.exe -i %1 -c:a pcm_s16le %1_COPY.wav
ffmpeg.exe -i %1 -c:v mjpeg -b:v 60000k -an %1_COPY.avi
ffmpeg.exe -i %1_COPY.avi -i %1_COPY.wav -shortest -map 0:0 -map 1:0 -c:v copy %1_FINAL.avi

Extract the audio, convert it to PCM. Remove the audio from the video, recompress to MJPEG. Combine separated audio and video files in to final product. Generates the 3-second-long-audio-track issue in Sony Vegas.


So Far The Only Working Solution


ffmpeg.exe -i %1 -c:a pcm_s16le %1_COPY.wav
ffmpeg.exe -i %1 -c:v mjpeg -b:v 60000k -an %1_COPY.avi

Extract audio and video separately, recombine them in VirtualDub. Tedious as hell, especially if I'm capturing footage for something large-scale. And I can't just convert the M2TS file in VirtualDub alone, either, as that generally seems to cause VirtualDub to crash.

If you need the full console output from FFMPEG, here's one such:

C:\Documents and Settings\Ryan>"F:\fraps\Roxio\ffmpeg.exe" -i F:\fraps\Roxio\2-1
7-2014_10.m2ts -c:v mjpeg -b:v 60000k -c:a pcm_s16le F:\fraps\Roxio\2-17-2014_1
0.m2ts_COPY.avi
ffmpeg version N-60572-gccc48b3 Copyright (c) 2000-2014 the FFmpeg developers
built on Feb 12 2014 22:01:44 with gcc 4.8.2 (GCC)
configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-av
isynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enab
le-iconv --enable-libass --enable-libbluray --enable-libcaca --enable-libfreetyp
e --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --ena
ble-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-l
ibopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libsp
eex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aa
cenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavp
ack --enable-libx264 --enable-libxavs --enable-libxvid --enable-zlib
libavutil 52. 63.101 / 52. 63.101
libavcodec 55. 52.101 / 55. 52.101
libavformat 55. 32.100 / 55. 32.100
libavdevice 55. 9.100 / 55. 9.100
libavfilter 4. 1.102 / 4. 1.102
libswscale 2. 5.101 / 2. 5.101
libswresample 0. 17.104 / 0. 17.104
libpostproc 52. 3.100 / 52. 3.100
[mpegts @ 02a5a960] PES packet size mismatch
Input #0, mpegts, from 'F:\fraps\Roxio\2-17-2014_10.m2ts':
Duration: 00:04:34.76, start: 1.200000, bitrate: 12179 kb/s
Program 1
Stream #0:0[0x1011]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 1
280x720 [SAR 1:1 DAR 16:9], 59.94 fps, 59.94 tbr, 90k tbn, 119.88 tbc
Stream #0:1[0x10f]: Audio: aac ([15][0][0][0] / 0x000F), 48000 Hz, stereo, f
ltp, 9 kb/s
[swscaler @ 02b46ca0] deprecated pixel format used, make sure you did set range
correctly
Output #0, avi, to 'F:\fraps\Roxio\2-17-2014_10.m2ts_COPY.avi':
Metadata:
ISFT : Lavf55.32.100
Stream #0:0: Video: mjpeg (MJPG / 0x47504A4D), yuvj420p, 1280x720 [SAR 1:1 D
AR 16:9], q=2-31, 60000 kb/s, 59.94 tbn, 59.94 tbc
Stream #0:1: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s16
, 1536 kb/s
Stream mapping:
Stream #0:0 -> #0:0 (h264 -> mjpeg)
Stream #0:1 -> #0:1 (aac -> pcm_s16le)
Press [q] to stop, [?] for help

And as its finishing encoding, it barfs up this:

[mpegts @ 02a5a960] PES packet size mismatch
[h264 @ 02c85400] error while decoding MB 20 9, bytestream (-6)
[aac @ 02a5eb00] Input buffer exhausted before END element found
Error while decoding stream #0:1: Invalid data found when processing input
[h264 @ 02c85400] concealing 2909 DC, 2909 AC, 2909 MV errors in B frame
video:2010209kB audio:51344kB subtitle:0 data:0 global headers:0kB muxing overhe
ad 0.034985%

I've posted about this on the FFMPEG forums (see here (http://ffmpeg.gusari.org/viewtopic.php?f=11&t=1292)) but as of yet nobody there has gotten back to me with a solution, and it's been more than a week. I uploaded 1.2gb example file if anyone needs something to play around with. It's split in to three parts:

https://www.dropbox.com/s/5g15ahhu4968m1r/2-11-2014_04.7z.001 (499mb)
https://www.dropbox.com/s/j1msqzf2fi2w3a4/2-11-2014_04.7z.002 (499mb)
https://www.dropbox.com/s/hx0140kafjefifq/2-11-2014_04.7z.003 (153mb)

If that's too big, I can try to upload something a little smaller, instead.

I have my fingers crossed that somebody here can help me out. The Roxio was "cheap", but it was still $120 out of my pocket.

[G_M_C]

Have you trioed eac3to to demux your capture and remux video/audio into another container converting audio if needed? See: http://forum.doom9.org/showthread.php?t=125966

[TheLostSkeleton]

Have you trioed eac3to to demux your capture and remux video/audio into another container converting audio if needed? See: http://forum.doom9.org/showthread.php?t=125966

Isn't that what I'm trying to do with FFMPEG? Separating the audio and video and then recombining them again ("Attempted Solution 4")? What's more, unless I have Nero 7 installed, the codecs used by eac3to are the same ones used by FFMPEG, it seems.

Then again, looking more in depth at the eac3to documentation, I feel a little over my head. I tried feeding it an M2TS file to get it to convert to WAV and it told me

"Unfortunately eac3to doesn't contain an (LGPL) AAC decoder."

So either I track down a highly-specific version of Nero from eight years ago, or... what, exactly?

[Sparktank]

Why not save to AVI with lossless video compression, like UT Video Codec Suite?
Limiting yourself to FFMPEG only is quite a drawback to NLE editing. Lossless video codecs should be VFW-friendly, which in turn, should be friendly for Sony Vegas.

It shouldn't take much at all to index the recorded stream and then frameserve it to VirtualDub via Avisynth for a lossless video format to edit in Sony Vegas.

[TheLostSkeleton]

Why not save to AVI with lossless video compression, like UT Video Codec Suite?
Limiting yourself to FFMPEG only is quite a drawback to NLE editing. Lossless video codecs should be VFW-friendly, which in turn, should be friendly for Sony Vegas.

It shouldn't take much at all to index the recorded stream and then frameserve it to VirtualDub via Avisynth for a lossless video format to edit in Sony Vegas.

I'd say I'd be worried about disk space, but these MJPEG files are getting rather large; a 1gb M2TS file can easily turn in to 6-10gb AVI, and that's probably comparable (or worse, even) to a lossless file.

I'm not especially familiar with Avisynth, though. I poked at it once a few years ago but never bothered to learn much of anything about it; how much setup are we talking, here? Part of the reason I was using FFMPEG is because I've actually written a (strictly for personal use) FFMPEG front-end before and I had planned on finding encoding settings that "worked" and adapting them to that, automating the conversion process so I could do large groups of files easily.

From what I remember about Avisynth, wouldn't I have to write a separate script file for each individual video?

[G_M_C]

Isn't that what I'm trying to do with FFMPEG? Separating the audio and video and then recombining them again ("Attempted Solution 4")? What's more, unless I have Nero 7 installed, the codecs used by eac3to are the same ones used by FFMPEG, it seems.

Then again, looking more in depth at the eac3to documentation, I feel a little over my head. I tried feeding it an M2TS file to get it to convert to WAV and it told me

"Unfortunately eac3to doesn't contain an (LGPL) AAC decoder."

So either I track down a highly-specific version of Nero from eight years ago, or... what, exactly?

Seems to me the audio is the main problem. On AAC - Faad2 / FAAC has worked for me in the past. Dunno if it works with eac3to, maybe you can use FAAD2/FAAC on its own? Convert audio to a flavour that vegas can handle, and mux that together with the original AVC-video into a container that works for vegas icw the audio-format you chose.

[Maccara]

I've had some issues with Movie Studio (the consumer version of Vegas Pro) opening m2ts files (not similar to yours, though).

However, h264+aac in mp4 has worked fine instead of m2ts, so you could just try remuxing with ffmpeg -i file.m2ts -c:v copy -c:a copy file.mp4 (based on your output, the files are h264+aac inside m2ts).

If that doesn't work, you could try reconverting the audio (as that seems to be the issue) just with ffmpeg -i file.m2ts -c:v copy -c:a aac -strict -2 -b:a 240k -r:a 48000 -async 1 file.mp4.

I'm using -async 1 because you seem to have some issues with the audio. I've noticed ffmpeg does not deal with any start packet mismatch or gaps in audio (or vfr files) properly (missing audio, video/audio not in sync etc) without async specified.

Not the ideal solution (conversion is not lossless), but give it a try in case that would at least get you forward. If that works, you could try with pcm_s16le (with the async option) if that helps.

[TheLostSkeleton]

I've had some issues with Movie Studio (the consumer version of Vegas Pro) opening m2ts files (not similar to yours, though).

However, h264+aac in mp4 has worked fine instead of m2ts, so you could just try remuxing with ffmpeg -i file.m2ts -c:v copy -c:a copy file.mp4 (based on your output, the files are h264+aac inside m2ts).

If that doesn't work, you could try reconverting the audio (as that seems to be the issue) just with ffmpeg -i file.m2ts -c:v copy -c:a aac -strict -2 -b:a 240k -r:a 48000 -async 1 file.mp4.

I'm using -async 1 because you seem to have some issues with the audio. I've noticed ffmpeg does not deal with any start packet mismatch or gaps in audio (or vfr files) properly (missing audio, video/audio not in sync etc) without async specified.

Not the ideal solution (conversion is not lossless), but give it a try in case that would at least get you forward. If that works, you could try with pcm_s16le (with the async option) if that helps.

Hmmm.

Upon trying to copy video and audio, FFMPEG yelled at me to use -bsf:a aac_adtstoasc because of malformed data. Doing so did the trick, but... now both options you provided generate an even more frustrating issue: Sony Vegas has a history of not liking MPEG4 in general, and I was surprised when it imported the M2TS video track so well. Using your options, while I get an audio and a video track in Sony Vegas, I get nasty looking video corruption (example (http://i.imgur.com/We2B2O2.png)) or, more often than not, Vegas just claims the video track is completely black.

I feel like I'm going a little crazy, though; using -async, FFMPEG magically stopped returning errors upon finishing encoding, and I generated a few files that actually opened in Vegas "fine" by adding it to the existing commands I used in the OP (though they were missing the last 15 seconds of audio). Except... then that mysteriously stopped happening and now it's generating errors again, even though I'm using identical commands.

[mpegts @ 02a5a9e0] PES packet size mismatch
[h264 @ 035edae0] error while decoding MB 20 9, bytestream (-6)
[aac @ 02b60020] Input buffer exhausted before END element found
Error while decoding stream #0:1: Invalid data found when processing input
[h264 @ 035edae0] concealing 2909 DC, 2909 AC, 2909 MV errors in B frame
frame=16470 fps= 99 q=31.0 Lsize= 103540kB time=00:04:34.77 bitrate=3086.9kbits
/s
video:95247kB audio:7987kB subtitle:0 data:0 global headers:0kB muxing overhead
0.295922%

and the AVI files its encoding have the 3 second audio problem, with or without -async

I am completely lost.

[TheLostSkeleton]

Okay, so, the black MP4 problem was caused because Vegas was latching on to Quicktime's MP4 decoder.

Now that I've disabled that, it's using Vegas' own internal MP4 decoder (mcmp4plug2.dll), which "works"... but now the M2TS->MP4 files break up like that example image (http://i.imgur.com/We2B2O2.png).

The good news is, I figured out how to get it to stop spitting out AAC errors after encoding; I think I mixed up the order of some commands before. This gives me no errors about AAC encoding (but mpegts still complains about a "PES packet size mismatch"):

"F:\fraps\Roxio\ffmpeg.exe" -i %1 -c:v copy -c:a copy -flags +global_header -async 1 -bsf:a aac_adtstoasc %1.mp4

I could probably get rid of the global header thing, that was just something I added on a whim just to see if it would do anything.

Recompressing just using the -v:c mpeg4 doesn't work, x264 is one of those codecs that will take HOURS for a clip that's only a few minutes long... hmmmm.

It's super round-about, but it seems like I can copy everything to MP4, which VirtualDub has a much easier time opening and dealing with... and from there I guess I could recompress it to XVID. For whatever reason, Sony Vegas doesn't like the version of XVID used by FFMPEG, but it likes whatever version VirtualDub feeds in to. It used to be, back with my old 480i capture card, I'd take the raw captures and when I deinterlaced them/filtered them for clean-up, I'd also recompress them to XVID at max bitrate.

It's still slow, but 35 minutes to compress a 4 minute clip is a lot better than 2+ hours.

Not a clean solution, but it... works, I guess. :(

[stratocaster]

I do not know much about this encoding stuff, I simply gave a try to encode your 16min44 1.2Gb m2ts file into your requirements
which, I guess are AVI M-JPEG, WAV-sowt(pcm16)
encoding went flawlessly with no error whatsoever.
I didn't go through the entire 16 min, I just did 60 sec with an 84Mb output result. it took barely a minute.
so.. here's the output for your eyes and mostly for your vegas tests..
http://cs0001.free.fr/-/2-11-2014_04.AVI

[TheLostSkeleton]

I do not know much about this encoding stuff, I simply gave a try to encode your 16min44 1.2Gb m2ts file into your requirements
which, I guess are AVI M-JPEG, WAV-sowt(pcm16)
encoding went flawlessly with no error whatsoever.
I didn't go through the entire 16 min, I just did 60 sec with an 84Mb output result. it took barely a minute.
so.. here's the output for your eyes and mostly for your vegas tests..
http://cs0001.free.fr/-/2-11-2014_04.AVI

You didn't let it encode long enough. For whatever reason, the problems with audio in Sony Vegas only come up if you let it encode over 2:46, and the mpegts/aac errors only come up if you let it encode the whole file.

[stratocaster]

I re-encoded the entire length of 16min44 into AVI M-JPEG, WAV-sowt(pcm16)
it took 11min20 on my VAIO laptop to complete the conversion.
No error of any kind during encoding or during playing the resultant output.
Perfect Audio Synch, 1.4GB output filesize zipped+split into 6 parts
http://cs0001.free.fr/--/2-11-2014_04.ZIP 42MB
http://cs0001.free.fr/--/2-11-2014_04.Z01 250MB
http://cs0001.free.fr/--/2-11-2014_04.Z02 250MB
http://cs0001.free.fr/--/2-11-2014_04.Z03 250MB
http://cs0001.free.fr/--/2-11-2014_04.Z04 250MB
http://cs0001.free.fr/--/2-11-2014_04.Z05 250MB

Couldn't test it with vegas as i have no intention to install vegas.