Inserting I-frames in h264 stream

[monochrome]

Hi all,

I have a Panasonic TZ7 / SZ3, which produces AVCHD Lite video. The problem I have is this: The video stream is a single I-frame followed by P-frames. This makes seeking in the video very difficult, as the decoder must render frames 0...X-1 in order to render frame X.

So my question is this: Is there any way to losslessly convert P-frames in an H264 stream to I frames? I am thinking about doing the normal decoding process, but each 100 frames or so, output the current fully-rendered frame as an I frame, and for all other frames just copy the P-frame data to the output stream.

Anyone has any ideas on how to do this?

[Asmodian]

I am pretty sure this is not possible with any tool available now.

It might be possible to write a tool that could do this but it would be very tricky to get it to work and I don't think you could get it to output a standard H.264 video.

Best option is to re-encode the entire video.

Edit: remember the p-frame isn't expecting to get the I-frame references but the old p-frame references which wouldn't be the same if you decoded the p-frame and re-compressed to an I frame. If you use a lossless mode I-frame it is not a standard stream (and still they probably aren't the same reference blocks in some way). I am sure there are other even more difficult problems with trying to do this.

[monochrome]

Ok, thanks. I've been bothered by the inability to trim the start of clips for a while...

[SquallMX]

For editing you should try a intermediate codec (Cineform, MJPG).

[Blue_MiSfit]

Or AVC Intra, which is free and will have better quality than MJPG. Simply use x264 in I-frame only mode, and set bitrate high, maybe 50 or 100mbps. This will be harder to decode in the NLE, but you should have a fast quad core if you're editing, right? ;)

~MiSfit

[monochrome]

For editing you should try a intermediate codec (Cineform, MJPG).

Or AVC Intra, which is free and will have better quality than MJPG. Simply use x264 in I-frame only mode, and set bitrate high, maybe 50 or 100mbps. This will be harder to decode in the NLE, but you should have a fast quad core if you're editing, right? ;)

Thank you both for your replies. I did look into transcoding using h264 lossless with only intra-frames, and that looks like a very good codec for final editing.

As for a "fast quad core" - I have an Asus Eee PC with an Intel Atom processor that may or may not have delusions of grandeur; all I know is that it is kinda slow when it comes to h264 encoding. Which brings me to why I'm doing this: I have a TZ7 and my laptop with me when I'm traveling. What I want to do is simply to roughly cut the clips I make while on the road - just so I don't fill up my harddrive unnecessarily (six weeks on the road generates a lot of bytes).

Avidemux in copy mode works fine, but the start of the clip ends up with severe macroblocking (as I've cut away many of the frames references by the P-frames).

Do any of you know if there is a way to see the P-frame references in a H264 stream? For example, if I knew that no P frame references a frame further back than 100 frames behind itself, then I can just cut 100 frames before the point where the interesting stuff starts, and be certain that I'll get no macroblocking.

BTW: Is there any limit at all as to what frame a P-slice may reference? Can a P-slice at frame 1000 reference any frame from 0-999, or just the last N frames?

[Blue_MiSfit]

:O saying an Atom is "kinda slow when ti comes to h.264 encoding" is a vast, epic understatement ;) A machine like this is a web / office only netbook . It's in no way a machine for working with digital media. But, if it's what you've got, it's what you've got.

If you just want to cut clips - use DGAVCIndex! It will work perfectly fine for such a simple task.

You can also use something like TSMuxeR or MKVMerge GUI to do this - but the procedure is a bit different (typing in timecodes instead of seeking on a timeline). You can't cut wherever you want without totally farking up the stream as you've seen, but these tools will only cut on I frames closest to your given timecodes (IDR?) so no worries.

~MiSfit

[monochrome]

A machine like this is a web / office only netbook . It's in no way a machine for working with digital media. But, if it's what you've got, it's what you've got.

Yep. But it is only for the road. Back home I have a Core2Duo. Not the best and fastest machine, but I'm just setting out to teach myself movie making, so as you say, it'll just have to do for now. This is all a bit of trying to see just how far the stuff I have will go - so your help is very appreciated. :thanks:

If you just want to cut clips - use DGAVCIndex! It will work perfectly fine for such a simple task.

I downloaded this one, but I couldn't find the "cut" function. I know that I can index movies and so make them searchable, but if I want to cut them, it seems like I have to run the indexed file through AVISynth and then recode it.

these tools will only cut on I frames closest to your given timecodes (IDR?) so no worries.

Yeah... see... this is where it gets ugly.There is only a single IDR frame. Frame zero. All others are P-frames... So it seems like farking up the stream is the only way to go, or recode.

[Blue_MiSfit]

Yikes! That's not very good! You're absolutely sure there are no other I frames anywhere in the stream?? Geesh..

Well, it's worth a try anyway. DGAVCIndex is pretty simple. Drop a supported file into it (transport stream, program stream, elementary stream), and then you can use the timeline or your keyboard arrows to seek throughout the stream. Use the "[" and "]" buttons to set in and out points, then file -> save project and demux video, and you will get elementary audio and video streams cut at your defined in an out points.

I'd be interested to see how DGAVCIndex can deal with a stream like you describe. We shall see!

If this route doesn't work, then you will probably have to do a full re-encode. I'd suggest indexing your source with FFMS2, and then using AviSynth to trim the regions you're interested in. Check out SoundOut to handle audio at the same time. Then you can use a very lightweight x264 setting like this to make AVC Intra streams.

#Example, for a rough guess at 100mbps CBR intra-only

x264 input.avs --bitrate 100000 --vbv-bufsize 100000 --preset fast --tune fastdecode --keyint 1 --min-keyint 1 --output "output.264"


Glad to be of some help.

~MiSfit

[monochrome]

Yikes! That's not very good! You're absolutely sure there are no other I frames anywhere in the stream?? Geesh..

Yep. Avidemux tells me I have (for example), one I-frame and 600 P-frames.

I'd be interested to see how DGAVCIndex can deal with a stream like you describe. We shall see!

It produces a ~400kb output file that can't be opened with anything I have.

I did manage to lossless-y cut the stream with Avidemux (but only when outputting to AVI). The resulting stream can then be recoded as H264 lossless (cqp = 0), intra-only for editing. I think that's going to be the route I take.

ffmpeg -i cut.avi -vcodec libx264 -vpre lossless_ultrafast -threads 0 -g 1 for_editing.avi

For 720p/25fps I get around 80Mbps.

I have no problems getting the resulting files into Blender for compositing and sequencing. So it would appear that everything works: Shooting, rough cutting, editing and final render. :)

Again, thanks for the help!

[Blue_MiSfit]

Awesome!

~MiSfit

[Asmodian]

It produces a ~400kb output file that can't be opened with anything I have.

Are you using Avisynth? You need to open the project file from DGAVCIndex using the DGAVCdecode.dll plugin for Avisynth. Avisynth will frame serve the decoded video as un-compressed I-frames to your editing application so you can cut where ever you want. You can also do editing in the Avisynth using scripts: (example)

AVCSource("file.dga", true)
Trim(100,0) #cuts the first 100 frames
Trim(100,2000) #keeps only frame 100 to frame 2000
BicubicResize(width, height) #to resize the output if wanted

Avisynth is very powerful once you get it setup and you can stop doing the lossless intermediate files if you just need to do simple editing, for complex editing in a GUI the decode lag from Avisynth might be too annoying. You can use multiple source files and do almost anything in Avisynth if you take the time to figure out how to write the script.