View Full Version : tokenizing chunks in a header-missing mp4-H264 bitstream (solved)
Nefast
2nd May 2007, 08:18
Hello all !
How can one find the beginning of a picture (i.e. the beginning of a specific NAL unit) in a H264 encoded bitstream, knowing that this information is commonly present in metadata (such as stco table of ISO 14496 standard), but assuming this metadata is unreadable or missing ?
I was hoping these NAL units where the IDR-type one, but in fact this is not true. Having read the UIT/H.264 specification, I haven't found a clue to this matter.
Someone help ? ^^
Thanks to all.
Nefast
2nd May 2007, 10:37
Let me turn the problem in an other way.
How X264 decides which NALU will be referenced as the first element of a chunk ?
or
How can I retrieve the first NALU of each chunk in the file ?
or
How can I reconstruct a stco table ?
and alternatly
Is it possible ?
In the file X264.c, from a recent release of the project X264, I'have found what seems to be the specific call to the function that interest me :
In encodeFrame, a picture (frame ?), is splitted in several NALUs and then, each NALU is written. At the end, a call for set_eop is made, altering the metadata to set the reference to the picture in the stco table.
As far as I understand all this, this mean that for each frame/picture encoded, a reference (the adress or offset of the NALU) is created.
Unfortunately, the sample files I am working on are not this way, ( the stco table of my mp4 H264 encoded file is referencing NALU that are not IDR-slice nor I-slice) and I'm beginning to wonder if I would one day understand where I am wrong.
The spec is clear on this, although it is non-trivial to implement. Refer to section 7.4.1.2.4: Detection of the first VCL NAL unit of a primary coded picture. That is the method I use in DGAVCIndex. AUD NALs are optional so you can't rely on them.
I suppose akupenguin will comment on the implementation in x264.
Nefast
2nd May 2007, 13:54
Well, thank you for your help, but I must say that I was already aware of this, and this is not solving my problem as in my sample file, frame_num is varying constantly (+1 increment for each NALU until reach of an IDR frame then zero and so on), and each NALU is not set as the first VCL NALU of a primary coded picture, although the section 7.4.1.2.4 specifies it is a sufficient condition for a NALU to be the first VCL NALU of a primary coded picture.
As far as I understand the whole thing, it would mean that all of the NALU in my file are "first VCL NALU of a primary coded picture", but when I try to reconstruct the stco table in that way, i.e. referencing all the VCL NALU, the file remains jammed. (I made sure the rest of the tables and structures remain coherent with these modification, so I'm sure the problem won't come from an other location).
Post a decent sized piece of your raw stream so we can analyze it.
You can have a full picture in one NALU. And if there are no intervening SEIs, SPSs, PPSs, etc., then you can have a stream of NALUs where each one is the first of a coded picture. Assuming your stream may be like that, is the issue then just understanding how the chunking is done for the container file? Your thread title asked about tokenizing pictures and I don't think there is an issue with that.
the file remains jammed
Please explain that.
Nefast
2nd May 2007, 14:21
Is this decent ?
Quote:
the file remains jammed
Please explain that.
Well, it is unreadable : VLC is crashing when I try to read it. Saying it excepts a I frame and doesn't get one.
That's not a raw stream, that's an MP4.
Again, you posted asking about tokenizing pictures in a raw stream.
Are you actually asking about how the raw stream is chunked for the MP4 container?
Nefast
2nd May 2007, 14:29
Yeah, I'm mistaking using 'raw', I'd rather say headerless. And yes, you're absolutely right, it's the beginning of each chunk I'm trying to retrieve.
Have you studied the appropriate container specification documents? 14496-12, 14496-14, 14496-15
Nefast
2nd May 2007, 14:36
Yep, I'm up to date with the 14496-12, but honestly, I was assuming it was enough to understand the chunking, so I didn't get the 14496-14. Does it worth purchasing it ? (I must say it is a bit expensive).
I haven't ventured into the container area yet, so I'm good for raw streams only at this time. :)
Maybe someone more knowledgeable of container issues will help you.
Nefast
2nd May 2007, 14:54
Well, thank you all the way. You helped me make things clear.
Drachir
2nd May 2007, 18:02
Yep, I'm up to date with the 14496-12, but honestly, I was assuming it was enough to understand the chunking, so I didn't get the 14496-14. Does it worth purchasing it ? (I must say it is a bit expensive).
14496-14 is no help to solve your problem.
mediator
2nd May 2007, 19:48
If you have lost moov/stco the only possibility to find frame starts is with a very sophisticated statistical analysis, or something like an attempt to "virtually" start parsing/decoding the bitstream at different positions. But all potential solutions still stay rather complex...
Why is that? Can't you demux the video and then re-containerize it in a normal manner?
neuron2: MP4 lumps all A/V data inteleaved together in mdat chunks, so it's not trivial to separate frames again, especially if it's headerless AAC audio. AVC is slightly easier as each NALU is size-prefixed. Also SPS, PPS and NALU size length are stored in stsd, so you'd have to reconstruct them as well.
Nefast
3rd May 2007, 08:32
neuron2: MP4 lumps all A/V data inteleaved together in mdat chunks, so it's not trivial to separate frames again, especially if it's headerless AAC audio. AVC is slightly easier as each NALU is size-prefixed. Also SPS, PPS and NALU size length are stored in stsd, so you'd have to reconstruct them as well.
Indeed, detecting AVC NALU beginning is quite easy using a custom bitmask. (size 4Bytes + NALU header 1Byte)
I've already successfully reconstructed headers of a MPEG4-VSP (DivX, XviD) because each frame can be referenced in the moov/stco table.
The problem with MPEG4-AVC is that you can't reference in moov/stco all the frame that are present in the mdat field, nevertheless the moov/stsz table gives the size of ALL the frames (or samples or NALUs in my case) of the mdat field.
That makes me think that only "special" frame must be referenced to allow the decoder to retrieve the reference frames it needs.
Nefast
3rd May 2007, 08:41
If you have lost moov/stco the only possibility to find frame starts is with a very sophisticated statistical analysis, or something like an attempt to "virtually" start parsing/decoding the bitstream at different positions. But all potential solutions still stay rather complex...
Are you saying that the moov/stco should list the offset of the beginning of each frame ?
In my case, the original file only stores in its stco table offset of some frames, and it works perfectly that way. The question is why this frame and not the other one ? I've crawled trough the NAL header and Slice header, but there is no hint that can differenciate referenced frame from non referenced ones.
It seems there is no logical way in the choice of referenced frames.
Perhaps it is the randomly seekable frames. They are identified by being either IDR frames or frames marked with a recovery point SEI.
Nefast
3rd May 2007, 14:15
Perhaps it is the randomly seekable frames. They are identified by being either IDR frames or frames marked with a recovery point SEI.
Unfortunately no. None of the 16 IDR frames my file contains is referenced in moov/stco.
Most of the NALU referenced in moov/stco are in fact predicted frames. The first one is a SEI NALU, and the fifth a SPS NALU, but all the twelve others are P frames.
Look at this, it can make things clear :
SEI <-- stco reference #1
SPS
PPS
IDR
P (*8)
IDR
P (*3)
P <-- stco reference #2
P (*5)
IDR
P (*5)
P <-- stco reference #3
P (*3)
IDR
P (*7)
P <-- stco reference #4
P
IDR
P (*9)
SPS <-- stco reference #5
IDR
P (*9)
IDR
P
P <-- stco reference #6
...
The only constant thing is that the numbers of VCL Slices between each stco reference is twelve.
:confused:
mediator
3rd May 2007, 17:52
An stco entry is usually started after switching tracks on multiplexing. I.e. if you mux 5 consecutive frames to track number 1 then there will be just one stco entry pointing to the first frame (or "access unit", like ISO likes to say). The offsets for the other frames are determined differentially via the respective sample sizes. If I remember correctly Annex A.7 or so of ISO 14496-12 tells how to use all those tables. There is not even a guarantee that a keyframe is directly pointed to by an stco entry.
Hopefully it gets a bit clearer now.
Nefast
4th May 2007, 10:02
Well, thank you, it opened a way in my mind to the solution : after several editing with an hexadecimal viewer, I finally found the answer : there is absolutely no link between how the frame are encoded and the chunk split. I managed to reference in moov/stco only the first of my frames with a correct moov/stsc table, and the file is readable. All my thanks to all of you.
:thanks:
A question is remaining : why X264 arbitrary decided to split my file in several chunks, as only one was sufficient ?
But this matter is not a real one for me such I can continue working on my recovery tool. :p
vBulletin® v3.8.11, Copyright ©2000-2026, vBulletin Solutions Inc.