avc editing (join two gops)

[tulup]

Hello,

let's say we have 2 separate GOP avc encoded. the first one is in IPPP.., and the second one is AVCHD like (IBBBP..). GOP actually from the same avc stream, the first one is re-encoded, so the stream parameters like size, interlaced etc. - are equal.

the software is able to normal decoding and play each gop separately. if we join the second one right after the first one (simply, concatenating two files), the stream become broken on the last P frame of the first GOP.

reading avc docs we found that frame_num and poc_lsb should be adjusted to resolve the problem.

and the question is: could anybody point me to the formula, how the encoder is calculated frame_num and poc_lsb? think i need to adjust the second GOP slice_header fields when concatenating two GOPs.

thanks,
a.

[neuron2]

The second GOP is open and so you cannot simply concatenate them like that and expect it to work.

Also, IP... may still be an open GOP if the I is not an IDR. So even if you have that as a second GOP, it may not work.

You may also have complications with different SPS/PPS's being activated for the two GOPS.

The formula you seek is in the AVC standard.

[tulup]

The second GOP is open and so you cannot simply concatenate them like that and expect it to work.

yep, it's non-sense. but we want to do it and see the result.


Also, IP... may still be an open GOP if the I is not an IDR. So even if you have that as a second GOP, it may not work.


we assume that the first GOP is always starts with IDR (as i wrote it's re-encoded part of the same stream). And the second one is may be or may be not.


You may also have complications with different SPS/PPS's being activated for the two GOPS.


yep, there were a several ones: progressive/interlaced - so we re-encode the first GOP to be interlaced too. comparing SPS and PSP, there are a few diffs still found (log2maxframenum_minus4 and lsb_minus4 not equal).


The formula you seek is in the AVC standard.


ok. we'll try to read it again and again.

1: IPPPP.. (12 frames)
frame_num++ each frame;
poc_lsb+=1 each slice in frame; +=2 each frame
last P frame: frame_num = 11; poc_lsb= (22, 23);

2: IBBP.. (12 frames)
frame_num increments once per 3 frames in stream order.
poc_lsb +1 to each slice in display order.

so tried we modify second GOP slice_header fields in several ways. like to continue the values frame_num from the first GOP (frame_num = 12, 13, 14) and to reset values poc_lsb to 0,1,2 (because 2 gop poc_lsb len in bits are lower and can't be 24 etc.).

the results is better but still decoder is going crazy right in the begining of the second GOP.

[neuron2]

the results is better but still decoder is going crazy right in the begining of the second GOP. Can't you step through it in the debugger and see what is m aking it crazy? And what do you mean by crazy? Something other than just the missing references you will run into by concatenating an open GOP?

[tulup]

we are using commercial Elecard AVC codecs, so trace is not avail.

The second GOP is open and so you cannot simply concatenate them like that and expect it to work.
by the way, forgot to mention. actually there are frames exists in the first GOP (they just re-encoded) which is referenced by B frames of the second one.

to define what is we call 'crazy', let's see the example in attachments (sorry, there was a attachment limits, so we used .7z to split files).

part0.zip - it's not actually zip content (extension modified cause of .zip limits is higher than others). it's avc elementary stream of the first (re-encoded ) GOP.

part1_00x.7z - it's .7z archive (the original one was part1.7z.00x, but wasn't able to attach file with extension .00x). it's avc elementary stream the second GOP.

If you can advice where to place merged stream with no limits on the size - i can show it too.

to see what is mean 'crazy' - just concatenate both files part1+part2.
we tried the software tsMuxerR (with default params) to join two files. join is working, but the result is not correct too.

If you know some other tool, or idea how to fix merged file, please let's know.

currelty we tried to modify the second GOP slice_headers, the result is better, but not the ideal one.

[neuron2]

Please just post a screenshot. Or describe it in words.

You are not going to get a clean video doing what you are doing. You have to reencode the second GOP.

[Dmitry Vergheles]

Andrey could you post the linsk to the files you mentioned?

[Dmitry Vergheles]

Please just post a screenshot. Or describe it in words.

You are not going to get a clean video doing what you are doing. You have to reencode the second GOP.

Hi Neuron,

Could you please clear up why second GOP is necessary to be re-encoded? Is is not the task to re-encode all both gops but to make frame accuracy editing.

To re-encode a part of a GOP the first frame is located in and join that data with other not altered AVC data. And the main question is what is there some specific aspects that have to be obeyed to do that. May be someone can suggest some tool to make frame accuracy AVC editing wiht partially re-encoding?

[neuron2]

Could you please clear up why second GOP is necessary to be re-encoded? Because it is open.

[Dmitry Vergheles]

Because it is open.

:D Russian fameous writer Anton Checkoh wrote - "That can't be so, becuase it can never be so"

GOP openness only means that first B-Frames are predicted from some frames from previous GOP. other GOP data can be decoded independenetely. Is not it?

[Sergey A. Sablin]

:D Russian fameous writer Anton Checkoh wrote - "That can't be so, becuase it can never be so"

GOP openness only means that first B-Frames are predicted from some frames from previous GOP. other GOP data can be decoded independenetely. Is not it?

for the only case when no more then 1 reference frame is used - else even P-frames will reference pictures from previous GOP.

[neuron2]

In addition to what Sergey said, which is correct, I add that when I say the GOP needs to be recoded I obviously mean that only the portions off the GOP that *need* recoding need to be recoded. Would Checkoh like that too?

It's just a semantic difference here in what we mean by "the GOP needs to be recoded".

BTW, I've never seen a stream in the wild where P frames uses references in the previous GOP. Have you seen them, Sergey?

[Sergey A. Sablin]

BTW, I've never seen a stream in the wild where P frames uses references in the previous GOP. Have you seen them, Sergey?

I bet you can easily generate such with x264 - multiple reference frames and single flash frame + scenecut would result in a non-IDR I-frame and next P-frame would reference something before this flash I-frame.

[Dmitry Vergheles]

Dear Sergey,

Thank you very much for detailed explenation.
Yes I aware that according to standard any frame can reference to another frames tob not a part of same GOP.

But as Neuron mentioned have you seen such kind of stream by yourself?

Assume P frame of GOP #n references to P Frame of GOP #n-1, if the reference P frame(GOP #n-1) is re-encoded is it a problem for a decoder?

[Sergey A. Sablin]

But as Neuron mentioned have you seen such kind of stream by yourself?
I have no time to analyse every existent stream just for this purpose, I've just said, that such widely used encoder as x264 can produce such stream with common settings. It's just obvious that such streams exist - you just need to find them.

Assume P frame of GOP #n references to P Frame of GOP #n-1, if the reference P frame(GOP #n-1) is re-encoded is it a problem for a decoder?

as I can understand you're talking about situation like this (which is different from original question):
GOP(n-2) GOP(n-1) GOP(n) GOP(n+1)
where something from GOP(n) reference something from GOP(n-1) and you gonna reencode only GOP(n-1). In this case even if you'll keep original GOP structure of GOP(n-1) - the picture pixels will be different from originals and thus GOP(n) will be distorted. And error (theoretically) will propagate until full refresh, i.e. IDR.

[neuron2]

I thought x264 does not use non-IDR I frames. Am I wrong about that?

My software treats I as IDR for seeking. I have never seen nor had reported a case of bad decode after a seek attributable to that. That tells me that P frames following I frames that reference frames in the previous GOP must be extremely rare in the wild.

[Sergey A. Sablin]

I thought x264 does not use non-IDR I frames. Am I wrong about that?

afaik scene-change I-frames in x264 are non-IDR.

[kemuri-_9]

I thought x264 does not use non-IDR I frames. Am I wrong about that?

if --scenecut is enabled and triggers on a frame where it is < --min-keyint distance from the last IDR frame, it will use a non IDR I-frame for the frame's slicetype.

[Dmitry Vergheles]

as I can understand you're talking about situation like this (which is different from original question):
GOP(n-2) GOP(n-1) GOP(n) GOP(n+1)
where something from GOP(n) reference something from GOP(n-1) and you gonna reencode only GOP(n-1). In this case even if you'll keep original GOP structure of GOP(n-1) - the picture pixels will be different from originals and thus GOP(n) will be distorted. And error (theoretically) will propagate until full refresh, i.e. IDR.


Yes you are right, I'm asking about this situation.Please cplear up one more thing:) Assume in MPEg-2 inside one GOP we re-encode one of P frame that is a reference for the next P one.

Is it the same situation for decoder as for AVC if we re-encode P frame (from Gop n-1) that is a reference for some P of (GOP n )?

Will the probable errors caused by difference of re-encoded frame similar for MPEG-2 and AVC cases?

[Sergey A. Sablin]

Yes you are right, I'm asking about this situation.Please cplear up one more thing:) Assume in MPEg-2 inside one GOP we re-encode one of P frame that is a reference for the next P one.

Is it the same situation for decoder as for AVC if we re-encode P frame (from Gop n-1) that is a reference for some P of (GOP n )?


yes.

Will the probable errors caused by difference of re-encoded frame similar for MPEG-2 and AVC cases?

errors would be different, cause intra prediction in AVC uses pixels even from inter MBs for prediction, thus errors might propagate not even from frame to frame, but even inside one frame from top-left corner towards bottom-right corner. In contrast for MPEG-2 intra DC prediction uses only intra MBs for prediction, so errors might propagate only from frame to frame. Thus structure of distortion would be different. However it all depends on the re-compression quality - the closer re-encoded frame to original, the less visible distortion you'll get.

PS. why don't you use skype? you're online - I see you ;)

[Dmitry Vergheles]

yes.



errors would be different, cause intra prediction in AVC uses pixels even from inter MBs for prediction, thus errors might propagate not even from frame to frame, but even inside one frame from top-left corner towards bottom-right corner. In contrast for MPEG-2 intra DC prediction uses only intra MBs for prediction, so errors might propagate only from frame to frame. Thus structure of distortion would be different. However it all depends on the re-compression quality - the closer re-encoded frame to original, the less visible distortion you'll get.


Thank you for the explanation.

PS. why don't you use skype? you're online - I see you ;)

By two reasons.
1 - the time did not come, but I will - be bet :)
2 - I'm sure your knowledges would be quite usefull for the somebody else on this forum

[tulup]

Because it is open.

we re-encode the very first B frames of the second GOP, so the GOP is closed. also, frame_num and pic_order_cnt_lsb were adjusted according to the AVC specification.

the original stream (http://solveigmm.com/files/utils/avc/00013.MTS)

the first GOP (re-encoded GOP # 1) (http://solveigmm.com/files/utils/avc/partOne.h264)

the result (re-encoded GOP #1 + original adjusted GOP #2) (http://solveigmm.com/files/utils/avc/parts2.h264)

could you please guys, try to play `the result` stream (maybe in frame step mode) and report decoding problem/broken frames.