Encoding everything vs. Appending parts via mkvmerge

[eriler]

Hi,

I'm wondering if anyone knows if there are any downsides to using the mkvmerge append function to append encoded h.264 video parts together, rather than encoding it all as a whole in AviSynth (It all has the same framerate of course).

I.e. I encode the
*Beginning as one part
*Intro as one part
*Main content/episode as one part
*Ending/credits/preview as one part

And then append it all together in mkvmerge.

Rather than the avisynth solution of
part1 = trim(x, x)
part2 = trim(x, x)
part3 = trim(x, x)
part4 = trim(x, x)
part1+part2+part3+part4

[LoRd_MuldeR]

If you are encoding with 2-Pass mode, it obviously means that the encoder can only distribute the bits within each part, rather than being able to distribute them within the whole movie.

For example, something like "ending credits" can usually get away with a very low bitrate. But if you encode it in parts, then those bits cannot be moved to "main content", where they might be used better.

Each part will get the same average bitrate (unless you adjust it manually for each part). Of course these considerations do not apply, if you are encoding in CRF mode...

[eriler]

Yes, if i were to use the append function i would only use it on movies/episodes where i've used:
*The same x264 settings on every part.
*CRF and not CBR.
*Single Pass encoding.
*The only thing that would some/most cases differ between the parts would be the deinterlacing in the avisynth script.

So using the append function has no downsides at all, no skips or stutter in the video, or anything like that? It would be the same as if i'd have encoded all the parts and joined them in the avisynth script?

[poisondeathray]

Of course there is a chance it might skip, stutter or not join when appending with mkvmerge

You can reduce the chance of non union by encoding with --stitchable . That doesn't help you ifyou've already encoded without that

[Atak_Snajpera]

or you can just combine .264 raw files with plain copy /B chunk1.264+chunk2.264 combined.264

[eriler]

Thank you all for your replies. I'll give --stitchable a chance and see how it fares.

[mandarinka]

or you can just combine .264 raw files with plain copy /B chunk1.264+chunk2.264 combined.264

This used to work fine, but at some point this started to break after muxing into mkvmerge, if encoding settings differed (it would screw timecodes badly). So nowadays I recommend to use mkvmerge's append function instead, just to be safe.

I only encountered this issue with streams concatenated with copy /B when doing it with streams with somewhat different settings, but still, better to be safe (note that modern mkvmerge has an option to fix timestamps, that thing could maybe prevent such issues).

BTW, if you want to be extra safe, you can output to raw/ES format (.264) with x264. That puts headers at every keyframe IIRC, unlike MP4 and MKV output. Streams with the headers like that are much more robust when concatenating and cutting even without the --stitchable parameter and even when the encoding options are different.

Even more robustness is probably possible by giving different --sps-id to segments, but I don't really understand that tool much and it is probably unnecessary.

[hello_hello]

I regularly encode a video in sections and append them together with MKVMergeGUI because I use QTGMC a fair bit and I run it in single threaded mode. Pretty much any script which includes QTGMC gets split in half so I can encode the two halves at the same time.
Using --stitchable is a must.

[Atak_Snajpera]

In my distributed encoding mode i use simple copy /b to join raw .264 streams. In this case i don't thing that switch is 'a must' .

[sneaker_ger]

Mkvmerge not throwing error messages when importing "copy /b" files does not mean they are valid but there shouldn't be any problems when settings are exactly the same. Always use --stitchable otherwise like hello_hello said.

[Atak_Snajpera]

Settings are obviously the same and I have never experienced any negative effects after muxing. Mkvmerge does not complain. Muxed file plays fine on software and hardware players. Not a single user complained about muxing error.

[hello_hello]

I recall one day I was happily appending, the next...... not so much.

It does depend whether you're appending raw avc or MKVs where the encoder is writing the output directly to MKV. I discovered that while testing before starting this thread (http://forum.doom9.org/showthread.php?p=1639768#post1639768) where I first inquired about not being able to append successfully, and sneaker_ger alerted me to the newly added --stitchable option.

I found raw avc would append if CRF encoding was used (and the same settings) but not necessarily when using 2 pass encoding. I don't normally use 2 pass encoding myself but the person asking the appending question in this thread (http://forum.doom9.org/showthread.php?p=1619425#post1619425) was. It appears something was changed in the x264 encoder (between versions 2216 and 2230) which effected the ability to append files and the --stitchable option was added a little later to fix it.

https://mailman.videolan.org/pipermail/x264-devel/2013-July/010167.html
"Stops x264 from attempting to optimize global stream headers, ensuring that different segments of a video will have identical headers when used with identical encoding settings."

When appending files I'd use --stitchable just to be safe, but maybe when it comes to appending raw avc I'm being unnecessarily paranoid. Personally I always output to MKV rather than raw avc as it allows me to preview the encoded video before the encoding has completed, so for me --stitchable is a must.

[hello_hello]

BTW, if you want to be extra safe, you can output to raw/ES format (.264) with x264. That puts headers at every keyframe IIRC, unlike MP4 and MKV output. Streams with the headers like that are much more robust when concatenating and cutting even without the --stitchable parameter and even when the encoding options are different.

I've been output directly to MKV for.... forever..... and not had a problem playing encodes (or appending since --stitchable arrived) but is outputting to raw AVC technically a better way to do it, or after muxing is the end result the same?

[sneaker_ger]

Just outputting to raw AVC is not safe, mkv simply does not allow multiple headers within a single track. Always make sure to have matching headers either via --stitchable option or 100% matching settings. If you follow that rule it doesn't matter whether you append the raw bitstreams or the mkvs.

[mandarinka]

That might be possible, but I don't think that mkvmerge discards those headers when muxing later. .264 output does give you more robustness for cutting/splicing (even when your result gets muxed to matroska).

[sneaker_ger]

That might be possible, but I don't think that mkvmerge discards those headers when muxing later.

Mkvmerge does not discard any headers but that doesn't mean the file is compliant. This is not an mkvmerge limitation but a limitation of the mkv spec. Also, headers might not be written at every GOP meaning that only linear playback will work flawlessly. x264cli does discard headers when using mkv output (it's not based on mkvmerge).