Processing MP4

[Ghitulescu]

I have a bunch of MP4 files from a camera that splits them every 5 minutes. File sizes some 600M each.

Generally, I can use several tools and GUIs to CLI tools, like mkvmerge, which I currently use. The result is the same.

I also discharge the audio, which is not important and poses some issues by itself.

The files can be seen on hardware players, like BDplayers, TVsets and stuff, perfectly only when they are the first of the series. Depending on the software player, this issue may also present.
There is another warning, issued when joining the second and any subsequent file to the first one.
The codec's private data does not match. Both have the same length (26) but different content. Please make sure that the resulting file plays correctly the whole time.

Now to the problem:
mkvmerge issues a warning, namely
The AVC video track is missing the 'CTTS' atom for frame timecode offsets. However, AVC/h.264 allows frames to have more than the traditional one (for P frames) or two (for B frames) references to other frames. The timecodes for such frames will be out-of-order, and the 'CTTS' atom is needed for getting the timecodes right. As it is missing the timecodes for this track might be wrong. You should watch the resulting file and make sure that it looks like you expected it to.

This does not cause any trouble (as MKV) on VLC, even older versions, and can be also correctly seen eg in Solveig Trimmer, which I used for cutting out the parts I need therefrom.

The file DOES however plays "strobo" every half second or so (I think at those P frames, see the warning above).

I find it strange that the remuxed video from MP4 into MKV loses this synch.

This is the problem.
What would be the solution to stamp the frames with the correct TC in those files in order to have a fluent playback?
Or, alternatively, can one change the process flow to avoid this issue (new/other tools, some options)?

[sneaker_ger]

Post sample files, then we can test. Maybe it's something that can be fixed in mkvtoolnix. Is the camera using some kind of proprietary splitting function that need merging similar to VOB on DVD (on bitstream level)?

[mkver]

My guess: If you look at the pts in the resulting matroska file, they are monotonically increasing, but they shouldn't be, because the original file uses b-frames so that the display order and coding/sotirng order don't match. Solution: Extract the H.264 track and remux it with mkvmerge. This will fix the timestamps. (The reason is that mkvmerge has two input modules for H.264: the "framed" one and the "unframed" one. The former expects a H.264 track in mp4 form (i.e. the NAL units are prefixed with their length, allowing easy parsing; and the samples have correct pts and dts), the latter expects an annex B stream (i.e. NAL units are prefixed by start codes and the beginning and end of NAL units and of "access units" (everything belonging to one frame/field of video; a sample in mp4 terminology) have to be determined; also the timestamps have to be derived from the bitstream information). The H.264 tracks inside mp4 and Matroska use the framed one (this is fast, because it is essentially only copying), the other container (avi, elementary stream, transport stream) use the unframed input module. This used to be very slower, but beginning with version 18, its speed has been greatly improved.)
PS: Uploading a sample might nevertheless provide more information.

[Ghitulescu]

I don't have the original files, they have been joined already, and the originals deleted for lack of space.
I did not notice this issue, because both the raw files, as well as the joined ones, played PERFECTLY on many computers on VLC.
They "jump" however on HW ones.

So, is it still possible to repair the joined files (recover the TCs) via conversion into a raw H264 file according to annexB, then remuxing them back in the final format?

[sneaker_ger]

Maybe, maybe not. You won't get around both providing samples and testing in your players. We can only guess. If you are not willing to do either I suggest we close the thread.

[Ghitulescu]

I am willing, but I do not have the originals anylonger, for I discovered the error only after joining them. As I said, on VLC they played perfectly.

I can provide samples from the joined files, if any help. If told how to obtain them.

[sneaker_ger]

Best would be to upload the complete file. Or at least cut a sample around a join point (e.g. from 00:04:30 to 00:05:30) with mp4box and/or mkvmerge.

Did you try mkver's suggestion with the demux then mux?

[Ghitulescu]

I have found a short original clip, some 20M, I'll upload it together with a joined file that has been cut 10 seconds around the joining point, over this weekend.
I do not have any subscription to a specific file sharing service, any good suggestion (preferably without intrusive ads)? bear in mind the samples are around this size or more (the bitrate is enormous compared to the video quality, unfortunately).

Retested several processed samples (including the suggestions above), this time only one two HW players, a Pioneer 430 and a Popcorn C200.: the short original sample works on all of them, the processed ones "jump" more (C200) or less (430). Even more, on C200, the remuxed (both in M2TS and MKV) samples play ok the very first seconds then start juddering. Apparently they play again ok after some FWD, to start judder again after a few seconds.

[Ghitulescu]

The raw file (https://www.datafilehost.com/d/b5213179)...

[Ghitulescu]

The cut section (https://www.file-upload.net/download-12872324/FILE171025-1133311-0021-001.mkv.html) of the processed file...

[poisondeathray]

Writing application : mkvmerge v5.3.0
Muxing mode : Header stripping


Try a newer mkvmerge to remux

There was a period in time where header stripping was the default setting and it caused many problems with players, especially HW players. It might be a contributing factor

[sneaker_ger]

Try these samples (https://mega.nz/#!hsUVnQ7Z!jSqpRGLSUCNDedCDNeQ17CHTsr49DHcpQbM6pF9mGKo) on your players.

[Ghitulescu]

So I tried them, only on the two I have at hand, the Pio 430 and the Popcorn C200.

The results are:
on C200 all files flickered except for M-P.
On the Pio, the first 6 (A-F) as well as O have been rejected/skipped because of the unsupported audio. Sample M was skipped without any error message. All other have been played OK with the exception of sample Q where it stuttered at the joining point.

So it appears that only sample N and P were accepted by both players. Over this week I'll try other HW players, as I get them or to them.

[sneaker_ger]

SampleP.mp4 is just FILE171104-175508.MP4 renamed. So I wonder why you uploaded a sample that plays fine?

SampleN is FILE171104-175508.MP4 extracted by "mp4box -raw 1 FILE171104-175508.MP4", then muxed using l-smash "muxer -i FILE171104-175508_track1.h264?fps=30000/1001 -o SampleN.mp4".

Samples C-F do not have audio at all.
Try again: https://mega.nz/#!Z5MilKjJ!OHHwDTlSOPfP7uEvFzxRQQFfVQVSSpUbRzwNRqSqKAo

But I don't know why the samples that don't work/flicker don't work/flicker. Stream looks normal to me. Maybe someone with more knowledge (like mkver) can see why.

[Ghitulescu]

SampleP.mp4 is just FILE171104-175508.MP4 renamed. So I wonder why you uploaded a sample that plays fine?

Because you asked me to provide one. Luckily, because it was so small, I forgot to delete it, so it's the only original file I have.

[Ghitulescu]

SampleN is FILE171104-175508.MP4 extracted by "mp4box -raw 1 FILE171104-175508.MP4", then muxed using l-smash "muxer -i FILE171104-175508_track1.h264?fps=30000/1001 -o SampleN.mp4".

It may be a stupid question, but why did you put 30000/1001? tsMuxer complained about the lack of fps information, and I used what was written upon the retail box: 30fps not 29.97.

[sneaker_ger]

I didn't give it much thought. 30/1.001 is more often used than 30.

[Ghitulescu]

Also the samples R and S jitter.

However, I used the follwing command (generic coding, I used it from another thread (https://forum.doom9.org/showthread.php?t=174644) of mine):
ffmpeg -i 01.mp4 -c copy -bsf:v h264_mp4toannexb -f mpegts in1.ts
ffmpeg -i 02.mp4 -c copy -bsf:v h264_mp4toannexb -f mpegts in2.ts
ffmpeg -i "concat:in1.ts|in2.ts" -c copy output.ts
pause

and it worked.
The ffmpeg version is 3.3.2, I do not know how old it is but it works on both XP and 10. That is my problem, most new versions work only on newer OSes and no longer on XP or 2000/w2k I still have.

[Ghitulescu]

It worked for the sample I have (the original).
It does not correct, unfortunately, the already joined files.

[sneaker_ger]

It does not correct, unfortunately, the already joined files.
Are you talking about "FILE171025-1133311-0021-001.mkv"? Problems start at 00:00:10,566?

[mkver]

The SPS in the CodecPrivate in FILE171025-1133311-0021-001.mkv is 67 42 00 28 BB 40 3C 01 12 F2 A0; at 10095ms there is another SPS/PPS transmitted inband. It is 67 42 01 28 BB 40 3C 01 12 F2 A0. The difference is 00 vs 01 in the third byte. Said 01 is actually against the spec: The first six bits of this byte are constraint_set0_flag,..., constraint_set5_flag and the last two bits are two reserved bits which should be equal to zero (in future versions of the H.264 standard, this can change). The semantics for this field includes: "Decoders shall ignore the value of reserved_zero_2bits." Maybe your players don't do this or are otherwise confused by this.

[Ghitulescu]

The SPS in the CodecPrivate in FILE171025-1133311-0021-001.mkv is 67 42 00 28 BB 40 3C 01 12 F2 A0; at 10095ms there is another SPS/PPS transmitted inband. It is 67 42 01 28 BB 40 3C 01 12 F2 A0. The difference is 00 vs 01 in the third byte. Said 01 is actually against the spec: The first six bits of this byte are constraint_set0_flag,..., constraint_set5_flag and the last two bits are two reserved bits which should be equal to zero (in future versions of the H.264 standard, this can change). The semantics for this field includes: "Decoders shall ignore the value of reserved_zero_2bits." Maybe your players don't do this or are otherwise confused by this.

This sounds Kilngonian to me, however, even if I instruct eg tsmuxer to insert SPS the resulting file still judders. That is ONLY for the processed files.

However, the files joined with annexb as per fictitious example above there is no issue.

Unless I reencode them, I see so far no solution to this.

EDIT: vlc tells me that the fps is 29.918fps.... strange... very strange...

[Ghitulescu]

Probably the last piece of information:

if I mux the old, processed files with tsmuxer and specify the 30/1.001 fps and insert SPS all players play the resulted M2TS file as desired.

Therefore I can save my files, by recomputing their time codes.

Damn camera.

[sneaker_ger]

@Ghitulescu
Please compare to test mkver's findings: https://mega.nz/#!4kchDKSY!bxNC7tdsKuXB57OnrxiFxQNoIr8xNezAw1wTKMxmeRM

P.S.: Doesn't x264 use reserved bits for a watermark?

[mkver]

Here are a few hypothesis about what happens:
1. Your devices don't like Matroska. To Test this hypothesis, remux your m2ts file that works to Matroska (with ffmpeg (mkvmerge has special code in place that deals with changing SPS/PPS and is thereforeand not exactly what we need for this task)) and see what happens.
2. Your devices don't like the thing I talked about earlier. The most direct way to test this is by deleting the SPS/PPS in-band:
ffmpeg -i input.mkv -c:v copy -bsf:v h264_mp4toannexb,remove_extra=keyframe output.mkv
3. You have let tsMuxer not only insert SPS/PPS, but also change the bitstream in different ways, depending on its handling of SEI and VUI data:
a) If I select "Do not change SEI and VUI data" (meta file "MUXOPT --no-pcr-on-video-pid --vbr --vbv-len=500 V_MPEG4/ISO/AVC, "I:\FILE171025-1133311-0021-001.mkv", fps=29.970, contSPS, ar=As source, track=1, lang=und"), tsMuxer inserts SPS/PPS (from the extradata) in front of keyframes until it reaches the frame with the defective SPS/PPS in it. This SPS/PPS is kept and no further SPS/PPS are inserted. Testing whether such a file works (as a m2ts) would shed further light on whether it is the issue I spoke about earlier.
b) If I select one of the other two options ("Insert SEI and VUI data if absent" or "Always rebuild SEI and VUI data"; for this input file they lead to identical output files) will change the SPS/PPS in a number of ways: The bitstream now contains timing information in the SPS and the hrd parameters (which are used for checking the stream for conformance with the profile and level it indicates; decoders just decode and therefore ignore these values (usually)) are inserted. Also the corresponding hrd SEI messages are inserted. It also replaces the defective SPS/PPS with the normal SPS/PPS (and this is the last SPS/PPS it inserts (this is actually a bug in tsMuxer)). Option b) is the default option, so if you used this, the fact that it works is consistent with the hypothesis that they don't like the broken SPS/PPS. It could also be that they like the bitstream timing infos.
4. Your file has a fps of 29.918, a very unusual framerate. Maybe your devices want a more standard framerate.
[Edit]: @sneaker_ger: This is something entirely different. x264 doesn't play with this field (doing this would be a very bad move: After all, in the future these fields might really mean something). Instead it does the following: If you have a SODB (can have an arbitrary length in bits), you are supposed to make a RBSP (has a byte length) by appending a single bit equal to one and then appending bits equal to zero until the length in bits is divisible by 8, i.e. until you have bytes. x264 doesn't only use zeros in the last step. Decoders don't complain about this, but some certifiers do.

[Ghitulescu]

Here are a few hypothesis about what happens:
1. Your devices don't like Matroska. To Test this hypothesis, remux your m2ts file that works to Matroska (with ffmpeg (mkvmerge has special code in place that deals with changing SPS/PPS and is thereforeand not exactly what we need for this task)) and see what happens.
2. Your devices don't like the thing I talked about earlier. The most direct way to test this is by deleting the SPS/PPS in-band:
ffmpeg -i input.mkv -c:v copy -bsf:v h264_mp4toannexb,remove_extra=keyframe output.mkv
3. You have let tsMuxer not only insert SPS/PPS, but also change the bitstream in different ways, depending on its handling of SEI and VUI data:
a) If I select "Do not change SEI and VUI data" (meta file "MUXOPT --no-pcr-on-video-pid --vbr --vbv-len=500 V_MPEG4/ISO/AVC, "I:\FILE171025-1133311-0021-001.mkv", fps=29.970, contSPS, ar=As source, track=1, lang=und"), tsMuxer inserts SPS/PPS (from the extradata) in front of keyframes until it reaches the frame with the defective SPS/PPS in it. This SPS/PPS is kept and no further SPS/PPS are inserted. Testing whether such a file works (as a m2ts) would shed further light on whether it is the issue I spoke about earlier.
b) If I select one of the other two options ("Insert SEI and VUI data if absent" or "Always rebuild SEI and VUI data"; for this input file they lead to identical output files) will change the SPS/PPS in a number of ways: The bitstream now contains timing information in the SPS and the hrd parameters (which are used for checking the stream for conformance with the profile and level it indicates; decoders just decode and therefore ignore these values (usually)) are inserted. Also the corresponding hrd SEI messages are inserted. It also replaces the defective SPS/PPS with the normal SPS/PPS (and this is the last SPS/PPS it inserts (this is actually a bug in tsMuxer)). Option b) is the default option, so if you used this, the fact that it works is consistent with the hypothesis that they don't like the broken SPS/PPS. It could also be that they like the bitstream timing infos.
4. Your file has a fps of 29.918, a very unusual framerate. Maybe your devices want a more standard framerate.
Ok, my last day of work for this year so I can focus on these files.

Thanks for your time, guys.

(.... After all, in the future these fields might really mean something).
While in some standards, indeed, some reserved fields could have been modified in time, to my knowledge, these reserved fields NEVER changed in ANY windows (or Microsoft) own products - they prefer to redefine yet another structure, yet another scheme, yet another system instead of reusing the good (as compatible) old ones.

[Ghitulescu]

4. Your file has a fps of 29.918, a very unusual framerate. Maybe your devices want a more standard framerate.

Yeah, this was my first thinking, when I noticed this strange fps. After remuxing to 29.97, the file is played without any hiccups.

There is another error/warning you guys may know about.

From time to time (most probably at the joining point), DTS has a hiccup: the first value is always 126000, the second differ
https://forum.doom9.org/attachment.php?attachmentid=16163&stc=1&d=1513929715

[mkver]

Yes, but as has been told, your tsMuxer does more than just changing the framerate. In order to isolate the root cause you should really test the file produced by 2. above.
By the way: H.264 is not Microsoft's creation (they have probably done a little bit, but MS was actually behind VC-1, a competitor to H.264). And in the first version of the H.264 standard there were only three bits used for constrains and accordingly five bits reserved.
Can't you upload your pic somewhere else?

[Ghitulescu]

I will do the tests at home. That means they have to wait a bit.

Secondly, I am not really interested in MKV or MP4. I have MP4 files that I cannot use as I like (for instance both players from Pioneer, one generation apart and made by the same manufacturer, which is Sharp, play differently the same MP4 file or sequence of files).

Therefore I wanted to join them (as per #1 the camera I had had split the videos in 5 minutes blocks) and make them universally viewable. I decided that .TS or .M2TS (with properly formed H.264 video) would be the best choice.

This is where I got into difficulties. I have no issues cutting videos, remuxing, authoring etc. videos from a true camcorder (Panasonic), from a true Sat recorder, but I always had issues with files coming from sources like these (bordcamera, mediatheques and stuff) which were never thought to be played or used outside tablets/smartphones/computers in short SW players.

PS: I only meant that sometimes the reserved bits of fields are left unused for good, but there are also cases where they had been indeed reused by redefining their meaning.

[mkver]

Just a question: Are you still interested in this? Or weren't you at home the last two weeks and couldn't test my hypothesis?

[Ghitulescu]

I was 2 weeks away.
I am still interested in this, but I was expecting first for the photos to be approved.

[Ghitulescu]

Test done.
The MKV obtained by changinf the fps from the M2TS (obtained as per post #18 (https://forum.doom9.org/showpost.php?p=1827771&postcount=18) from MP4) works.

I will soon do the SPS/PPS test, although I processed this way the MP4 and saw no difference.


I am glad I sorted it out.

[Ghitulescu]

Done with the second command line, no difference - all players play them flawlessly.

I strongly believe it was the weird fps the one that caused this stuttering...

[sneaker_ger]

I strongly believe it was the weird fps the one that caused this stuttering...
That doesn't align with your reports from post #13.

Did you test the samples from #24?

[Ghitulescu]

I tested all the samples provide, unless they were skipped for non-supported reasons (since I am not on a computer, the name and extension were not available in the info fields, so I used some lists to match the files - also some of the players took the files sequentially as in the FAT, others ordered them by name, a mess).

Consequently, the only solution that works was presented above:
1. obtain the time-corrected joined file
ffmpeg -i 01.mp4 -c copy -bsf:v h264_mp4toannexb -f mpegts in1.ts
ffmpeg -i 02.mp4 -c copy -bsf:v h264_mp4toannexb -f mpegts in2.ts
ffmpeg -i "concat:in1.ts|in2.ts" -c copy output.ts
pause
the number and names of the files depending on the real conditions.
2. change the fps in tsmuxergui to 30000/1001, forcing the SSP/PPS insertion and to be sure set the DAR to 16/9.

The resulted file works on every player, HW or SW, I tested.