Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 AVC / H.264

Reply
 
Thread Tools Search this Thread Display Modes
Old 22nd April 2018, 20:12   #1  |  Link
NailBomber
Registered User
 
Join Date: May 2015
Location: Russia
Posts: 17
Generate a video with same settings as an existing video in order to join them

I have a H264 video to which I want to attach a little blank video.
I would like to avoid re-encoding a video and just remux it instead.
I tried different settings using ffmpeg x264 to generate a compatible blank video and got a result with seemingly same parameters (as reported by ffprobe and MediaInfo).
However when trying to join both videos MKVToolNix complains that "The codec's private data does not match".

I actually pretty much gave up on the idea, but before doing that wanted to hear from pros on this forum.

I downloaded a trial of CodecVisa to get more information, but it doesn't show anything in 'Stream Summary'.
NailBomber is offline   Reply With Quote
Old 23rd April 2018, 06:38   #2  |  Link
kuchikirukia
Registered User
 
Join Date: Oct 2014
Posts: 476
Don't use ffmpeg, use the x264 version they used.

If the blank video doesn't have to be an exact length you could also see if there's a section in the video you have that's blank from keyframe to keyframe, cut it out, and paste it where you want it.

If nothing you try works and your playback situation works with ordered chapters or linked mkvs (MPC-HC or VLC), you could just leave your created blank clip as a separate mkv and join it with chapters or segment linking.

Last edited by kuchikirukia; 23rd April 2018 at 06:42.
kuchikirukia is offline   Reply With Quote
Old 24th April 2018, 11:50   #3  |  Link
hubblec4
Matroska find' ich toll
 
Join Date: Apr 2008
Posts: 1,370
Hi,

Nice to read that other user knows about ordered chapters and segment linking... I'm impressed.

@NailBomber
You will find Matroska-Linking features in my chapterEditor.
hubblec4 is offline   Reply With Quote
Old 24th April 2018, 22:40   #4  |  Link
NailBomber
Registered User
 
Join Date: May 2015
Location: Russia
Posts: 17
Unfortunately I have no idea what x264 version was used.
Blank video have to be exact length.

I have no experience with segment linking, but looking through some tutorials it looks like the files won't be encapsulated into a single mkv, right? Like they will still be there.
Looks like I'll have to wait for AV1 encoder to mature enough and re-encode.
NailBomber is offline   Reply With Quote
Old 24th April 2018, 22:48   #5  |  Link
sneaker_ger
Registered User
 
Join Date: Dec 2002
Posts: 5,565
Quote:
Originally Posted by NailBomber View Post
MKVToolNix complains that "The codec's private data does not match".
That does not necessarily make them incompatible.

Quote:
Originally Posted by NailBomber View Post
I downloaded a trial of CodecVisa to get more information, but it doesn't show anything in 'Stream Summary'.
Click on the first frame in the time line. Then you should be able to see headers/SPS/PPS.
sneaker_ger is offline   Reply With Quote
Old 25th April 2018, 00:44   #6  |  Link
NailBomber
Registered User
 
Join Date: May 2015
Location: Russia
Posts: 17
After joining resulting file has correct duration, but when viewed appended video turns gray and has some timing issue.

Quote:
Originally Posted by sneaker_ger View Post
Click on the first frame in the time line. Then you should be able to see headers/SPS/PPS.
That's true, I'm just stupid. Going to investigate.
NailBomber is offline   Reply With Quote
Old 25th April 2018, 02:51   #7  |  Link
kuchikirukia
Registered User
 
Join Date: Oct 2014
Posts: 476
Quote:
Originally Posted by NailBomber View Post
Blank video have to be exact length.
What are you trying to do?
kuchikirukia is offline   Reply With Quote
Old 25th April 2018, 17:04   #8  |  Link
NailBomber
Registered User
 
Join Date: May 2015
Location: Russia
Posts: 17
I have several videos of one and the same broadcast.
They have filenames like "video_2017.11.23_15.32.10.ts". Next one will be like "video_2017.11.23_16.10.56.ts" and so on.
Videos have varying time gaps between them.
I'm trying to join them together while respecting the time flow and actual duration of a full broadcast ( (last_video.datetime + last_video.duraton) - first_video.datetime ).
If I simply remux and join them then the resulting video will have a wrong time flow and smaller duration.
So the idea was to make a Python script that would generate and insert blank videos of calculated length and then remux it all into a single video.
Like: video1 + blank1 (duration = video2.datetime - (video1.datetime + video1.duraton)) + video2 + blank2 (duration = video3.datetime - (video2.datetime + video2.duraton)) + ...

I got busy with sudden problems, so didn't have time to experiment for now.
NailBomber is offline   Reply With Quote
Old 29th April 2018, 22:34   #9  |  Link
NailBomber
Registered User
 
Join Date: May 2015
Location: Russia
Posts: 17
Finally did some more testing. Turns out that settings are largely irrelevant (for MKV concatenation that is).
It doesn't matter if attached video has different number of slices, references, b-frames, framerate... it will work well anyway.
The reason I was getting problems is because I was attaching an MKV produced by FFMPEG.
This command:
Code:
ffmpeg -loop 1 -i "image.png" -t 10.000 -vcodec libx264 -pix_fmt yuv420p -r 24 output.mkv
produces an MKV that's causing problems.
But if I do it like this:
Code:
ffmpeg -loop 1 -i "image.png" -t 10.000 -vcodec libx264 -pix_fmt yuv420p -r 24 output.h264
mkvmerge -o output.mkv output.h264
produced MKV is fine (even though MKVToolNix still complains).
What's going on here? Is FFMPEG's MKV muxing bugged?
I noticed that an MKV produced by FFMPEG loses "Frame Rate" and "Average Bitrate" information in Stream Summary->Stream Statistics in CodecVisa.
NailBomber is offline   Reply With Quote
Old 29th April 2018, 23:29   #10  |  Link
sneaker_ger
Registered User
 
Join Date: Dec 2002
Posts: 5,565
Quote:
Originally Posted by NailBomber View Post
Is FFMPEG's MKV muxing bugged?
No. If you use ffmpeg to encode with libx264 directly to mkv container ffmpeg will write SPS/PPS only once. If you output to H.264 ES (.264 file) then the SPS/PPS will be repeated for every keyframe. Then decoder gets fed when seeking to different parts of the file even if the parts are not compatible. But if the other part does not have repeated SPS/PPS there can still be problems with certain players. So best recommendation is still to encode with exactly matching parameters. Mkv wasn't designed with changing headers in mind.
sneaker_ger is offline   Reply With Quote
Old 30th April 2018, 13:47   #11  |  Link
mkver
Registered User
 
Join Date: May 2016
Posts: 197
There are principally two ways to concatenate almost any H.264 videos: The H.264 format stores its "extradata" in so called Sequence Paramter Sets (SPS) and Picture Parameter Sets PPS. Every SPS has an SPS-id (a number between 0 and 31; most encoders use 0 by default), every PPS has a PPS-id (in the range of 0 to 255, most encoders use 0 by default). Every PPS also references a SPS. A picture header (more exactly: a slice header) references a PPS by its PPS-id which means that the parameters that the referenced PPS and the paramters contained in the SPS referenced by said PPS are active for the decoding process. As soon as a decoder receives a SPS/PPS it stores this SPS/PPS; if it already had a SPS/PPS with the same SPS/PPS-id, that SPS/PPS is overwritten. (Of course, no SPS overwrites a PPS and vice versa.)
The SPS/PPS can be transmitted either "in-band" (along the usual video data) or "out-of-band" (in the case of Matroska: not in the bitstream, but in the header).
a) The first solution is the "in-band" solution: Transmit the keyframes with SPS/PPS (this is actually standard for e.g. TV broadcasts). This is actually what happens when you use the approach with elementary stream output as sneaker_ger has already said.
b) The second solution is the "out-of-band" solution: x264.exe accepts a parameter called --sps-id. With it you can set the SPS and PPS id used. If you set it to a value that no SPS-id or PPS-id of your other files use, then you can put both your new SPS/PPS in the extradata as well as the old one. They are compatible. But there is a problem: You have to resort to some hacks to actually join them, because mkvmerge has no functionality to actually detect if the extra data are compatible and doesn't join them if they are.
PS: Changing the currently active SPS is only allowed at IDR pictures.
PPS: In both cases, there is always the chance that some players don't like the SPS/PPS switch: If have seen VLC showing some damaged frames at the beginning of part with a different SPS/PPS (regardless of which of the two cases I pursued) while MPC-HC did not. After I increased the number of reorder frames for the second part (to the level used by the first SPS, thereby making the SPS more alike), playback was fine.
mkver is offline   Reply With Quote
Old 30th April 2018, 16:47   #12  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
Code:
as reported by ffprobe and MediaInfo
best use raw streams and h264_parse and compare it's output
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 1st May 2018, 00:05   #13  |  Link
NailBomber
Registered User
 
Join Date: May 2015
Location: Russia
Posts: 17
Thanks for responses. That explains problems I had.
The player I use (mpv) doesn't seem to have a problem with an "in-band" method.
Of course, it's desirable to have an exactly matching video.

I found options that seem to work for some settings (used x264 for tests):
Code:
Stream:
Slices = 4 (--slices 4)
IDR every 72 frame (--keyint 72)
No B-frames (--bframes 0)
Framerate = 24 (--fps 24000/1000)
Video Format = YUV420 (--output-csp i420)

SPS:
profile_idc = 100 (--profile high)
max_num_ref_frames = 1 (--ref 1)

PPS:
weighted_pred_flag = 0 (--weightp 0)
pic_init_qp_minus26 = 0 (--qp 26)
chroma_qp_index_offset = 0 (--subme 1 or 2)
I still don't know how to set some other settings.

Intra-frames:
Code:
original:
slice_type = 2
nal_ref_idc = 3
nal_unit_type = 5

generated:
slice_type = 7
nal_ref_idc = 3
nal_unit_type = 5
P-frames:
Code:
original:
slice_type = 0
nal_ref_idc = 3
nal_unit_type = 1

generated:
slice_type = 5
nal_ref_idc = 2
nal_unit_type = 1
Also original video has log2_max_frame_num_minus4 = 4.

What settings could be used to fix those?

Quote:
Originally Posted by Selur View Post
Code:
as reported by ffprobe and MediaInfo
best use raw streams and h264_parse and compare it's output
This one by drmpeg? (https://forum.doom9.org/showthread.php?t=167170)
NailBomber is offline   Reply With Quote
Old 1st May 2018, 03:29   #14  |  Link
mkver
Registered User
 
Join Date: May 2016
Posts: 197
Yes, that is the h264_parse mentioned. If you want to make the SPS and PPS the same, you should show us the SPS and PPS (as reported by e.g. h264_parse or ffmpeg's trace_headers).
mkver is offline   Reply With Quote
Old 1st May 2018, 09:01   #15  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
Yup, that the one. I use it in MKV Cutter to figure out the x264 settings needed for the reencoding of splitted gops.
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 1st May 2018, 16:25   #16  |  Link
NailBomber
Registered User
 
Join Date: May 2015
Location: Russia
Posts: 17
I attached h264_parse output of the original video.
Would be interested in knowing what settings affect what parameters.
Attached Files
File Type: zip h264_parse_output.zip (16.7 KB, 42 views)
NailBomber is offline   Reply With Quote
Old 2nd May 2018, 13:10   #17  |  Link
mkver
Registered User
 
Join Date: May 2016
Posts: 197
If you want help in this life, you should better upload that attachment (or better yet: a sample from your video) to another hoster. Approval of attachments is pretty slow at doom9.
mkver is offline   Reply With Quote
Old 2nd May 2018, 15:55   #18  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,558
Quote:
Originally Posted by NailBomber View Post
I still don't know how to set some other settings.

Intra-frames:
Code:
original:
slice_type = 2
nal_ref_idc = 3
nal_unit_type = 5

generated:
slice_type = 7
nal_ref_idc = 3
nal_unit_type = 5
P-frames:
Code:
original:
slice_type = 0
nal_ref_idc = 3
nal_unit_type = 1

generated:
slice_type = 5
nal_ref_idc = 2
nal_unit_type = 1
Also original video has log2_max_frame_num_minus4 = 4.

What settings could be used to fix those?
The differences in slice_type are just a good indication that they were made by different encoders; slice_type 5-9 mean exactly the same as 0-4 except 5-9 also mean, "oh, and by the way all the other slices in this frame will have this value too." Technically they could otherwise be different, though in practice this only appears with intra-refresh. nal_ref_idc should never affect decoding, only the priority level for dropping frames by the transport layer when congested.
foxyshadis is offline   Reply With Quote
Old 2nd May 2018, 23:14   #19  |  Link
NailBomber
Registered User
 
Join Date: May 2015
Location: Russia
Posts: 17
I uploaded the fragment I'm working with
https://drive.google.com/drive/folde...5y00KtQSzs8ZxZ

Quote:
Originally Posted by foxyshadis
The differences in slice_type are just a good indication that they were made by different encoders;
I had a thought that Chrome browser's encoder might've been involved somehow. From what I gathered it uses OpenH264 for encoding. However, tests with that encoder showed that it doesn't support High profile (although CABAC option is there)? Video I'm working with is High profile though, so if some other encoder was used then I have no clue.
NailBomber is offline   Reply With Quote
Old 11th May 2018, 14:33   #20  |  Link
MGarret
Registered User
 
Join Date: Feb 2007
Posts: 18
Why don't you just encode your blank video with x264 using --sps-id 1 and be done with it and not try to re-engineer x264 to what some other encoder is doing.

I've done that many times successfully, mixing x264 parts with segments from hardware encoders. Any recent and good decoder like LAV will decode it perfectly with no glitches. I even tried it right now, mixing main and high profile footage, field coding with mbaff coding, 3 Refs and 4 Refs segments etc, different log2 values etc and it's working. That feature is there exactly for that purpose, to join outputs from different encoders. Just be sure to encode your blank video to raw .264 and then use mkvtoolnix to append your two segments.
MGarret is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 21:15.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.