Processing MP4

[Ghitulescu]

I have a bunch of MP4 files from a camera that splits them every 5 minutes. File sizes some 600M each.

Generally, I can use several tools and GUIs to CLI tools, like mkvmerge, which I currently use. The result is the same.

I also discharge the audio, which is not important and poses some issues by itself.

The files can be seen on hardware players, like BDplayers, TVsets and stuff, perfectly only when they are the first of the series. Depending on the software player, this issue may also present.
There is another warning, issued when joining the second and any subsequent file to the first one.

Originally Posted by mmg

The codec's private data does not match. Both have the same length (26) but different content. Please make sure that the resulting file plays correctly the whole time.

Now to the problem:
mkvmerge issues a warning, namely

Originally Posted by mmg

The AVC video track is missing the 'CTTS' atom for frame timecode offsets. However, AVC/h.264 allows frames to have more than the traditional one (for P frames) or two (for B frames) references to other frames. The timecodes for such frames will be out-of-order, and the 'CTTS' atom is needed for getting the timecodes right. As it is missing the timecodes for this track might be wrong. You should watch the resulting file and make sure that it looks like you expected it to.

This does not cause any trouble (as MKV) on VLC, even older versions, and can be also correctly seen eg in Solveig Trimmer, which I used for cutting out the parts I need therefrom.

The file DOES however plays "strobo" every half second or so (I think at those P frames, see the warning above).

I find it strange that the remuxed video from MP4 into MKV loses this synch.

This is the problem.
What would be the solution to stamp the frames with the correct TC in those files in order to have a fluent playback?
Or, alternatively, can one change the process flow to avoid this issue (new/other tools, some options)?

[sneaker_ger]

Post sample files, then we can test. Maybe it's something that can be fixed in mkvtoolnix. Is the camera using some kind of proprietary splitting function that need merging similar to VOB on DVD (on bitstream level)?

[mkver]

My guess: If you look at the pts in the resulting matroska file, they are monotonically increasing, but they shouldn't be, because the original file uses b-frames so that the display order and coding/sotirng order don't match. Solution: Extract the H.264 track and remux it with mkvmerge. This will fix the timestamps. (The reason is that mkvmerge has two input modules for H.264: the "framed" one and the "unframed" one. The former expects a H.264 track in mp4 form (i.e. the NAL units are prefixed with their length, allowing easy parsing; and the samples have correct pts and dts), the latter expects an annex B stream (i.e. NAL units are prefixed by start codes and the beginning and end of NAL units and of "access units" (everything belonging to one frame/field of video; a sample in mp4 terminology) have to be determined; also the timestamps have to be derived from the bitstream information). The H.264 tracks inside mp4 and Matroska use the framed one (this is fast, because it is essentially only copying), the other container (avi, elementary stream, transport stream) use the unframed input module. This used to be very slower, but beginning with version 18, its speed has been greatly improved.)
PS: Uploading a sample might nevertheless provide more information.

[Ghitulescu]

I don't have the original files, they have been joined already, and the originals deleted for lack of space.
I did not notice this issue, because both the raw files, as well as the joined ones, played PERFECTLY on many computers on VLC.
They "jump" however on HW ones.

So, is it still possible to repair the joined files (recover the TCs) via conversion into a raw H264 file according to annexB, then remuxing them back in the final format?

[sneaker_ger]

Maybe, maybe not. You won't get around both providing samples and testing in your players. We can only guess. If you are not willing to do either I suggest we close the thread.

[Ghitulescu]

I am willing, but I do not have the originals anylonger, for I discovered the error only after joining them. As I said, on VLC they played perfectly.

I can provide samples from the joined files, if any help. If told how to obtain them.

[sneaker_ger]

Best would be to upload the complete file. Or at least cut a sample around a join point (e.g. from 00:04:30 to 00:05:30) with mp4box and/or mkvmerge.

Did you try mkver's suggestion with the demux then mux?

[Ghitulescu]

I have found a short original clip, some 20M, I'll upload it together with a joined file that has been cut 10 seconds around the joining point, over this weekend.
I do not have any subscription to a specific file sharing service, any good suggestion (preferably without intrusive ads)? bear in mind the samples are around this size or more (the bitrate is enormous compared to the video quality, unfortunately).

Retested several processed samples (including the suggestions above), this time only one two HW players, a Pioneer 430 and a Popcorn C200.: the short original sample works on all of them, the processed ones "jump" more (C200) or less (430). Even more, on C200, the remuxed (both in M2TS and MKV) samples play ok the very first seconds then start juddering. Apparently they play again ok after some FWD, to start judder again after a few seconds.

[Ghitulescu]

The raw file...

[Ghitulescu]

The cut section of the processed file...

[poisondeathray]

Writing application : mkvmerge v5.3.0
Muxing mode : Header stripping

Try a newer mkvmerge to remux

There was a period in time where header stripping was the default setting and it caused many problems with players, especially HW players. It might be a contributing factor

[sneaker_ger]

Try these samples on your players.

[Ghitulescu]

So I tried them, only on the two I have at hand, the Pio 430 and the Popcorn C200.

The results are:
on C200 all files flickered except for M-P.
On the Pio, the first 6 (A-F) as well as O have been rejected/skipped because of the unsupported audio. Sample M was skipped without any error message. All other have been played OK with the exception of sample Q where it stuttered at the joining point.

So it appears that only sample N and P were accepted by both players. Over this week I'll try other HW players, as I get them or to them.

[sneaker_ger]

SampleP.mp4 is just FILE171104-175508.MP4 renamed. So I wonder why you uploaded a sample that plays fine?

SampleN is FILE171104-175508.MP4 extracted by "mp4box -raw 1 FILE171104-175508.MP4", then muxed using l-smash "muxer -i FILE171104-175508_track1.h264?fps=30000/1001 -o SampleN.mp4".

Samples C-F do not have audio at all.
Try again: https://mega.nz/#!Z5MilKjJ!OHHwDTlSO...pUbRzwNRqSqKAo

But I don't know why the samples that don't work/flicker don't work/flicker. Stream looks normal to me. Maybe someone with more knowledge (like mkver) can see why.

[Ghitulescu]

Originally Posted by sneaker_ger

SampleP.mp4 is just FILE171104-175508.MP4 renamed. So I wonder why you uploaded a sample that plays fine?

Because you asked me to provide one. Luckily, because it was so small, I forgot to delete it, so it's the only original file I have.

[Ghitulescu]

Originally Posted by sneaker_ger

SampleN is FILE171104-175508.MP4 extracted by "mp4box -raw 1 FILE171104-175508.MP4", then muxed using l-smash "muxer -i FILE171104-175508_track1.h264?fps=30000/1001 -o SampleN.mp4".

It may be a stupid question, but why did you put 30000/1001? tsMuxer complained about the lack of fps information, and I used what was written upon the retail box: 30fps not 29.97.

[sneaker_ger]

I didn't give it much thought. 30/1.001 is more often used than 30.

[Ghitulescu]

Also the samples R and S jitter.

However, I used the follwing command (generic coding, I used it from another thread of mine):

ffmpeg -i 01.mp4 -c copy -bsf:v h264_mp4toannexb -f mpegts in1.ts
ffmpeg -i 02.mp4 -c copy -bsf:v h264_mp4toannexb -f mpegts in2.ts
ffmpeg -i "concat:in1.ts|in2.ts" -c copy output.ts
pause

and it worked.
The ffmpeg version is 3.3.2, I do not know how old it is but it works on both XP and 10. That is my problem, most new versions work only on newer OSes and no longer on XP or 2000/w2k I still have.

[Ghitulescu]

It worked for the sample I have (the original).
It does not correct, unfortunately, the already joined files.

[sneaker_ger]

Originally Posted by Ghitulescu

It does not correct, unfortunately, the already joined files.

Are you talking about "FILE171025-1133311-0021-001.mkv"? Problems start at 00:00:10,566?