Embedding a 264/AVC elementary stream in a .avi file

[dalleyg]

Is it possible to embed an existing 264/H264/x264/AVC elementary stream in an avi file? If so, how? I've been scouring the web and trying every suggestion I find for related questions, without success.

Things I've tried:

avidemux
avisynth
the whole lineup of Elecard products with easily-downloadable demos
Ulead VideoStudio 11 Plus

avidemux/avisynth don't seem to recognize the elementary stream files. Elecard XMuxer can output what it calls an AVI file, but normal programs don't seem to think it's a valid AVI file (GSpot thinks it's still an elementary stream with a very-wrong frame size, VirtualDub can't load it, VLC 0.8.6e crashes, ...).

Bigger picture (for those who care):
I have a Canon HG10 camcorder that records 1440x1080 progressive frames with a H.264 codec (despite the marketing claims about it doing 1920x1080). I am able to grab the .mts files (mpeg transport stream). I want to repackage this video into something that is readable via DirectShow, without doing any transcoding. I am doing video analysis so I want to avoid anything that requires transcoding.

Ulead can create AVIs for me, but it requires transcoding and it artificially introduces interlacing artifacts that are not present in the original stream.

[JoeShrubbery]

If you have a means of remuxing the video from the transport stream into a program stream (mts -> mp4) you could probably then use avidemux to remux again from mp4 to avi.

Keep in mind, you'll need an H.264 VFW codec on the system (enabling H.264 support in the FFDShow VFW settings could work)

[Dark Shikari]

M2TS can be opened with DirectShow.

You'll be much better off using MKV or MP4, since AVI doesn't support H.264 properly. You can get H.264 in AVI using avc2avi, I believe, but it will be very broken.

[dalleyg]

Thanks for the suggestions so far.

I've been using H.264 in avi for a few years now in other contexts (transcoding mpeg2 via VirtualDub and mencoder, using the old x264 VfW codec and more recently ffdshow). It seems to work fine.

If I use avc2avi, it thinks the frames are 1440x1088 (not 1440x1080). In VLC, the resulting file has junk for most of the frame and the top part of the frame is basically a sheared version of the frame. VLC crashes after 1 frame. This is the same result I get when using VLC on the original MTS file. MP4Box/YAMB is confused in the same way as avc2avi.

If I try loading any of the files in GraphEdit (the original MTS, the H.264 elementary stream, or the incorrectly-remuxed files), it turns out I do get a complete filtergraph. It uses ffdshow for decoding. Depending on which file I use, when I try to render the graph, it either just shows a blank frame, or GraphEdit crashes.

If I load the .mp4 output from YAMB into GSpot, I see the correct dimensions showing, but when trying to render it using the Microsoft API, I get a VFW_E_CANNOT_RENDER error.

It's starting to look like the problem is reversing the hackery from Canon.

[Sharktooth]

h.264 in avi is just an hackery as well.
it may work and i may not...
playback should be more or less always possible, while editing... fuggedaboutit.

[CruNcher]

Try something more professional then ffdshow like CoreAVC,Elecard/Mainconcept,Arcsoft,Cyberlink does it work then? and especialy wich parser you parse the .m2ts .avc with also try something professional here :)

I think also a good way is to use Elecard/Mainconcepts Demuxer and Demux the .m2ts you have now into a .ts or .mpg and then use Elecards/Mainconcepts Parser and tools it worked for me alot of times seems such H.264 .m2ts like it being remuxed to .ts and .mpg tough not many software can read those .ts .mpg files properly (Mplayer does Vlc i think not) (i think only Elecard/Mainconcepts own products) :D
Seeking is perfect in .ts aswell as .mpg (and no strange artifacting that appears when you mux into some other containers) :)

Tough H.264 in either .ts or .mpg is fully standard compliant (no b-frame problems no nothing) but as said not many support this (even commercial software) :)

[SeeMoreDigital]

AVC2AVI+GUI should be able to do what you require.

But as others have already mentioned, placing AVC video within the .AVI container is a definite "no-no" anyway....

[CruNcher]

Ahh and since some time there is even another possibility i think you could also try to demux it with Donalds DGAVCIndex it can also process .m2ts :)

[Dark Shikari]

As I said, VfW doesn't work with H.264, so you need to use DirectShow to access it.

[Sharktooth]

avc2avi will just do it (btw, i have also updated the GUI some days ago) but it's not recommended. as you can see (also said in the post above by D_S) VFW and h.264 do not work as they should...
just use another kind of container... and directshow or gstreamer.

[dalleyg]

Thanks again for the suggestions. I thought I'd summarize here what I've found.

avc2avi does indeed do the embedding.

It turns out that the .mts transport stream files can be decoded in DirectShow as Dark Shikari suggested if I use the Haali Media Splitter. Other splitters/demuxers such as Ulead's and ArcSoft's are partial implementations that just assume any M2TS streams are MPEG2. Those splitters have matched codecs that ignore the stream datatypes and just assume that the incoming data is exactly the type of H.264 they expect. Mixing and matching these decoders and splitters/demuxers often leads to application crashes.

To decode this video, I also needed to get a working H.264 decoder. At the present time, ffdshow doesn't handle all of the Canon HG10 videos correctly (they use PAFF encoding of b-frames with temporal prediction for the top fields and spatial prediction for the bottom fields). InterVideo's codec works, except it does deinterlacing and it the deinterlacer can't be disabled. This is bad because Canon stuffs 24p data into a 60i container using 2-3 pulldown. If deinterlacing is applied, the pulldown cannot be removed to restore the original 24p video. CoreAVC does the decoding correctly, its deinterlacer can be disabled, and when combined with the ffdshow raw video filter, pulldown reversal can be done to recover the original 24p video. Unfortunately, CoreAVC doesn't handle seeks correctly (the bottom fields show up wrong).

[SeeMoreDigital]

Hi dalleyg,

I've currently testing Ziova's CS615 hardware player. Are you able to provide a sample for testing?

Something with some movement coming toward you (like vehicles) would be good :)


Many thanks

[dalleyg]

Hi dalleyg,

I've currently testing Ziova's CS615 hardware player. Are you able to provide a sample for testing?

Something with some movement coming toward you (like vehicles) would be good :)

Many thanks

Here's a clip. It has vehicles moving sideways and a camera shake starting at about frame 26. That's when ffdshow first shows decoder errors.
http://people.csail.mit.edu/dalleyg/collaboration/avchd/hg10-pf24-clip2.mts

A longer description of this clip and a second similar clip can be found in the article: http://people.csail.mit.edu/dalleyg/collaboration/avchd/readme.html

[SeeMoreDigital]

Thanks...

Using the shorter hg10-24pf.MTS sample (http://people.csail.mit.edu/dalleyg/collaboration/avchd/hg10-24pf.MTS) as a source, it would seem the Canon HG10 produces pretty odd MPEG-4 AVC streams.

When I fed the source thru' DGAVCIndex it reports the following: -Stream Type: Transport [192]
Profile: High
Level: 4
Frame Size: 1440x1080
SAR: 4:3
Display Size: 1920x1080
Frame Rate: 29.970030 fps
Colorimetry: BT.709* [2]
Frame Structure: Fields (TFF)
Frame Type: B
Coded Number: 973
Playback Number: 973
Frame Repeats: 0
Field Repeats: 0
Bitrate: 6.477
Bitrate (Avg): 6.145
Bitrate (Max): 9.640
Audio Stream: 1100: AC3 2/0 256
Elapsed: 0:01:11
Remain: 0:00:00
FPS: 13.40
Info: One thing that sticks out is the "Frame Structure" is listed as being "Fields TFF" and not say, "MBAFF". Which seems to confuse my hardware player as it's unable to drop the interlaced frames (at: 2,7/12,17/22,27/32,37/etc, etc).

I'm sure Donald Graft (aka: neuron2) would be interested in this camera's streams!


Cheers

[dalleyg]

Thanks...

Using the shorter hg10-24pf.MTS sample (http://people.csail.mit.edu/dalleyg/collaboration/avchd/hg10-24pf.MTS) as a source, it would seem the Canon HG10 produces pretty odd MPEG-4 AVC streams.

When I fed the source thru' DGAVCIndex it reports the following: -Stream Type: Transport [192]
Profile: High
Level: 4
Frame Size: 1440x1080
SAR: 4:3
Display Size: 1920x1080
Frame Rate: 29.970030 fps
Colorimetry: BT.709* [2]
Frame Structure: Fields (TFF)
Frame Type: B
Coded Number: 973
Playback Number: 973
Frame Repeats: 0
Field Repeats: 0
Bitrate: 6.477
Bitrate (Avg): 6.145
Bitrate (Max): 9.640
Audio Stream: 1100: AC3 2/0 256
Elapsed: 0:01:11
Remain: 0:00:00
FPS: 13.40
Info: One thing that sticks out is the "Frame Structure" is listed as being "Fields TFF" and not say, "MBAFF". Which seems to confuse my hardware player as it's unable to drop the interlaced frames (at: 2,7/12,17/22,27/32,37/etc, etc).

I'm sure Donald Graft (aka: neuron2) would be interested in this camera's streams!


Cheers

Thanks for the extra analysis. I'll send it off to him. FYI, if I use the CoreAVC decoder with his AviSynth DeComb and FDecimate filters (via the ffdshow raw video filter), the pulldown removal works. Core fails on seeks right now, but at least sequential access works.

[LoRd_MuldeR]

You can get H.264 in AVI using avc2avi, I believe, but it will be very broken.

Sorry, but that is a common myth!
H.264 in AVI works works perfectly fine, when created properly. With all players I have tested so far (including MPlayer, VLC and DirectShow) playback was alright.
Playback on hardware/stand-alone players is a totally different question. AVI won' work on such players, that's for sure. You'll need MP4 (or MKV on some players).
Also you can re-mux the H.264 stream from AVI to another container at any time, using MKVtoolnix or MP4Box/YAMB.
Cutting H.264 in AVI container is problematic indeed! But cutting H.264 in MP4, MKV isn't any better at the moment - at least when using Avidemux to cut.
No idea of commercial tools...

[dalleyg]

Sorry, but that is a common myth!
H.264 in AVI works works perfectly fine, when created properly. With all players I have tested so far (including MPlayer, VLC and DirectShow) playback was alright.

Just to be clear... I believe my initial problems with AVI + DirectShow were due to ffmpeg's incomplete H.264 implementation. The version of VLC I am using 0.8.6e still does not work with AVI + Canon HG10 files. If I use a player where I can better control the decoders used (e.g. most DirectShow-based ones), I can get it to work with CoreAVC as long as I don't seek. The InterVideo decoder works as long as I don't mind it introducing deinterlacing artifacts (hint: it does matter for my applications). Some proprietary commercial apps like Sony Vegas yield the same results as the InterVideo decoder, even when I tell them to convert to 24fps video.

I have been using H.264 + AVI for years with videos I have encoded with x264 + VirtualDub or mencoder. They have worked fine for me, and cutting at I-frame boundaries seems to work (though I haven't tested that too much).

[Dark Shikari]

Sorry, but that is a common myth!
H.264 in AVI works works perfectly fine, when created properly. With all players I have tested so far (including MPlayer, VLC and DirectShow) playback was alright.Just because it works on a couple players which all use the same libraries doesn't mean its not broken.

[SeeMoreDigital]

Playback on hardware/stand-alone players is a totally different question. AVI won' work on such players, that's for sure. You'll need MP4 (or MKV on some players).Actually, they do on my Ziova CS615 (test unit).

That said, all my AVC in .AVI test samples are well below 2GB. And we all know how .AVI files tend to perform/act above 2GB.

For me, it's .MP4, .TS or .M2TS.... The .AVI container should have been dumped years ago. Sadly our friends over at DivX are keeping it alive :(

[rory83]

When I fed the source thru' DGAVCIndex it reports the following: -Stream Type: Transport [192]
Profile: High
Level: 4
Frame Size: 1440x1080
SAR: 4:3
Display Size: 1920x1080
Frame Rate: 29.970030 fps
Colorimetry: BT.709* [2]
Frame Structure: Fields (TFF)
Frame Type: B
Coded Number: 973
Playback Number: 973
Frame Repeats: 0
Field Repeats: 0
Bitrate: 6.477
Bitrate (Avg): 6.145
Bitrate (Max): 9.640
Audio Stream: 1100: AC3 2/0 256
Elapsed: 0:01:11
Remain: 0:00:00
FPS: 13.40
Info: One thing that sticks out is the "Frame Structure" is listed as being "Fields TFF" and not say, "MBAFF". Which seems to confuse my hardware player as it's unable to drop the interlaced frames (at: 2,7/12,17/22,27/32,37/etc, etc).

I'm sure Donald Graft (aka: neuron2) would be interested in this camera's streams!


Cheers
They are identical to the report from my UK Panasonic SD1 M2TS shots, other than the frame rate:
Stream Type: Transport [192]
Profile: High
Level: 4
Frame Size: 1440x1080
SAR: 4:3
Display Size: 1920x1080
Frame Rate: 25.000000 fps
Colorimetry: BT.709* [2]
Frame Structure: Fields (TFF)
Frame Type: B
Coded Number: 188
Playback Number: 188
Frame Repeats: 0
Field Repeats: 0
Bitrate: 11.792
Bitrate (Avg): 12.435
Bitrate (Max): 16.867
Audio Stream: 1100: AC3 3/2 384
Elapsed: 0:00:41
Remain: 0:00:35
FPS: 10.38
Info:

[SeeMoreDigital]

Hi Rory,

Are you able to provide a sample for hardware playback testing too please?

[LoRd_MuldeR]

Just because it works on a couple players which all use the same libraries doesn't mean its not broken.
Right!

But I think VLC Player has it's own AVI Splitter, independent from MPlayer's implementation.
The same applies to M$' DirectShow AVI splitter and MPC's internal AVI splitter.

Furthermore the H.264 decoder library shouldn't matter here, as long as the splitter reads proper H.264 bitstream out of the AVI container!
Well, at least ffdshow/libavcodec and CoreAVC don't complain ...

[rory83]

Hi Rory,

Are you able to provide a sample for hardware playback testing too please?
Yes, here, handheld telephoto hence shakey shot!
http://rapidshare.com/files/103628249/00005.m2ts.html

[SeeMoreDigital]

Thanks Rory,

These AVC streams are odd...

After extracting the elementary stream from the .M2TS container and re-muxing it into the .MP4 container, the output file becomes completely borked and unplayable in all of my software or hardware players :devil:

[Blue_MiSfit]

As a sidenote, your camera records anamorphic pixels. When converted to 1:1 PAR, you will get a 1920x1080 frame :)

~MiSfit

[SeeMoreDigital]

As a sidenote, your camera records anamorphic pixels. When converted to 1:1 PAR, you will get a 1920x1080 frame :)

~MiSfitActually no... A PAR value of 4:3 needs to be added to the 1440x1080 stream to obtain an 1920x1080 frame ;)

[Blue_MiSfit]

I stand corrected!! I get confused easily with PAR / DAR etc.

Thanks SeeMoreDigital

[SeeMoreDigital]

I stand corrected!! I get confused easily with PAR / DAR etc.As do many others...

At the end of the day PAR, DAR, SAR all offer the same thing. The ability to alter a pixels shape using different aspect ratio signalling (ARS) values.


Cheers

[rory83]

The vast majority of current hi-def consumer camcorders that claim 1920*1080 actually record at 1440, including some claiming 'full HD', whatever that may mean.! It's not only you that have trouble playing the raw streams, finding an editor that properly copes with them is not easy...

[DeathTheSheep]

Hey hey, guys. Check out my sig. It's the best example of AVC in .avi. Works like a charm, thanks to the efforts of all who contributed (lately especially our forum member BugMaster/MasterNobody). Works without hitch with VfW supporting applications (of which there are a darn huge number).

Also comes with VfW decoder. Check it out if you want to make AVC in AVI directly. YAMB also supports turning it into an MP4 with a single click.