Merging Encodes of a Movie spawned over multiple DVDs (LotR SEE)

[Shandra]

Ok, I am really not sure into which subforum this question should be placed, so I've selected the Newbies...

[Context]
* DVDs for Lord of the Rings Special Extented, each Movie is spawned over 2 DVDs (Ok, the following could possible be applied to true 2 sided DVDs of old or other similar cases)
* I wanted to merge the encodes of each movie into one file using mkvtoolnix-gui (video->x264, audio: keeping AC3 for Multichannel, AAC (QAAC) for 2.0, Subtitles (forced) [converted from vobsubs to srt with subtitle edit] + chapter indices).
* SRT Subtitles because I wanted to remove the black (Top/Bottom) Borders for the encoded videoframe and to have the possibility that a player could render the subs below the videoframe.

[Observation]
* Encoding both parts with the same setting for x264 and the same parameters within the avs script with MeGUI for uncroped Frame and Frame croped by 72/72 Top/Bottom (having in mind a Mod16 for the Result) resulted in (1st muxed each part in mkvtoolnix-gui/mkvmerge, then appending 2nd mkv to 1st mkv): Encode of full Frame (720x576) merged fine, croped encode delievers mostly grey macroblocks/garbage for the merged 2nd part of the video whence decoding.
* merging with Avidemux (Only the Videostreams and it would be nice if that one would honour PAR/DAR besides PAR 1:1) produced the same garbage for the 2nd part of the merged result.
* In another Encode -> Cropping 64/64 Top/Bottom went fine for the merge (as it was for the uncroped Encode)

[Question]
As Each Encode of Both Parts played Fine on its own, regardless wether Macroblocks of the Source where honoured or not (As we are all used to with encodes, regardless if we crop to MOD16 or just MOD2 and that cutting 8 Top+8 Bottom still resulted in MOD16 for the encoder); I don't get it why 2 Videostreams that where encoded with the same settings, but cropped the m2v (Edit: Ok, using d2v, but the source still is m2v within vob) source within macroblocks, can't be merged to be decoded properly, but if both encodes where croped honouring the macroblocks of the source could...

I really have no Idea wether this is a MeGUI, x264, mkvtoolnix or whatsoever problem - And as the 64&64 Crop is working I don't wan't any suggestions in regards of the encodes, but I am really puzzled why in this case of merging the encodes, it is needed to honour the 16x16 macroblock size of the source for both encodes.

[Edit:
P.S.: My 1st Encodes in this case are over a month old (was within an update-cycle of mkvtoolnix, the olderup2date version doesn't wanted to merge those croped files, the updated one accepted 'em) , and encountering the problem I tried multiple searches (here within the Forum and on the Web in general), but for the decoding problem of the merge I only found a few cases mentioning it, but not a real solution... The Idea of the multiples of 16 crop top/bottom just came within a private discussion 2 days ago, and as I am really/really and really not getting it why this is working, i've decided to post the "question"/proclaim the "observation" here - so please, don't ask for some logs or such for the described observations, all I can deliver is the mediainfo for the encodes of both parts I still have for some of the various encodes.]

[lansing]

You can't merge videos with different resolution, they have to be the same

[Shandra]

Iansing: Same AVS Script for both Parts of the Movie [Edit: As it is DVD(PAL) Source, both Sources are 720x576 and the crop for each of them is the same, so same resolution for the encodes to be merged; The following avs scripts are for the (filename/path) 1st part of the movie in question, it is the same script for the second (just replace _1 with _2 in Directory and Filename...)]

Encoding Both Parts with this setting have decoding problems after appending/merging...

# Set DAR in encoder to 64 : 27. The following line is for automatic signalling
global MeGUI_darx = 64
global MeGUI_dary = 27
LoadPlugin("D:\Video\MeGUI\tools\dgindex\DGDecode.dll")
DGDecode_mpeg2source("F:\Encode\LOTR1_1\LOTR1_1.d2v", cpu=6, info=3)
LoadPlugin("D:\Video\MeGUI\tools\avisynth_plugin\ColorMatrix.dll")
ColorMatrix(hints=true, threads=0)
#deinterlace
crop(0, 72, 0, -72)
#resize
#denoise

Whereas with this one it works without problems

# Set DAR in encoder to 16 : 7. The following line is for automatic signalling
global MeGUI_darx = 16
global MeGUI_dary = 7
LoadPlugin("D:\Video\MeGUI\tools\dgindex\DGDecode.dll")
DGDecode_mpeg2source("F:\Encode\LOTR1_1\LOTR1_1.d2v", cpu=6, info=3)
LoadPlugin("D:\Video\MeGUI\tools\avisynth_plugin\ColorMatrix.dll")
ColorMatrix(hints=true, threads=0)
#deinterlace
crop(0, 64, 0, -64)
#resize
#denoise

Edit: And as said - without croping the videoframe, it is also working fine whence merged....
(As said, I am really curious why Crops dividable by 16 for top/bottom are working for the merge and those who are splitting a macroblock of the source are not -> that one is in my idea of an encode (decoded/juvenile frames passed to the encoder) not making any sense at all!)

[lansing]

Originally Posted by Shandra

and those who are splitting a macroblock of the source are not -> that one is in my idea of an encode (decoded/juvenile frames passed to the encoder) not making any sense at all!)

I don't understand what you talking about. What exactly did you do?
And can you upload 2 encoded samples that you're having problem merging?

[Shandra]

Iansing: No I won't upload any (not even cuted down to a few seconds end/start part1/2) of my private DVDs encodes... What am I talking about? In Case my 1st Post may be open for interpretation withing the [context] part:
Here is what I've done: Encoding each part (Lord of the Rings SEE, 1 Movie split over 2 DVDs) with x264, muxing each part with mkvmerge (via mkvtoolnix-gui) (Video/Audio/Chapters/Subtitles) and then appending the mkv of the second part to the one of the first part within mkvtoolnix-gui (v17/v18 [*1]).

And here I will repeat myself again: This is going fine for: uncropped videoframe (720x576) and a crop by 64/64 (Top/Bottom), it results in garbage whence decoding the frames of the second part within the merge with a crop by 72/72 (Top/Bottom). And be it coincidence or not, describing it as: case (a) videoframe of the encode incorporates whole macroblocks of the source (uncropped and cropped by 64/64) -> no decoding problems within the merged file; case (b) videoframe of the encode incorporates a cut macroblock of the source (16x8) at top/bottom -> decoding problems within the merged file for the "second part" of the video.

And as said in my 1st post: I am not interested in a solution, as I am not really having a problem (Hey, what the heck - I could even just merge the m2vs and use the dvdsubs instead of srts) and just using the working 720x448 encode is fine for me. All I am is beeing curious and wanted to know if anyone knows an explanation for that observation (as far as my understanding is: the encoder would be served "juvenile" videoframes and therefore there is no rhyme or reason within this observation for me).
So yes, this may be the wrong subforum and maybe the mp4 AVC one would have been a better option, but to boil it down to the codec itself without naming AviSynth/MeGUI/MKVMerge/MP4Box and various Tools, I have no idea how to word that observation of mine into proper facts and techspeech to address it correctly (as I am doing this from a Users and not a Devs PoV) and considered it therefore a "Beginners" type of Question/Topic.

[1]: Audio/Subtitles/etc. have no influence and also the observation is independent of the container I use (mkv|mp4) for the merged Video.

[sneaker_ger]

H.264 uses header data (VPS/SPS) to decode frames. These can be different from encode to encode for example because of different resolution or source (x264 tries some optimization). When the image turns grey on the second part it often means the H.264 decoder is still using the header data of the first part. Matroska wasn't really designed for such changing header data. Therefore it is recommended to merge before encoding (e.g. in AviSynth using splicing). In case one doesn't want to splice before encoding it is crucial to have the encodings match 100%: same resolution, same settings and also to turn off header optimization (by setting "--stitchable" in x264). If encodes were already done in a wrong way sometimes better playback can be achieved by duplicating the VPS/SPS to each keyframe. AFAIK that's what the ffmpeg concat demuxer does.

[Shandra]

sneaker_ger: Ok, I will try the -stitchable flag for that encode again and yes, as it turned out in some tests (shortened clips) it was a pure coincedence that it happened with the 72/72 crop - I Have just done some quick encodes for shortened clips using various settings/resolutions and I am not getting any pattern that makes sense (work/doesn't work), and I can accept the "header data"/optimization explanation as a suitable "lies to children" one (But as merging the clips via AviDemux with the BuildIn MP4 Muxers (stream set to copy mode) had the same problems within the mp4 container) - and maybe some RTFM for the "stitchable" Option my provide some more Details, as somewhat I had hoped for some "rules of thumb" for the intent of merging encodes

P.S.: Thanks! I really wasn't aware of the -stitchable parameter, and the funny thing is: searching for information about it, resulted in finding all the Postings I tried to find before posting here; Aka The Chicken/Egg/ReverseEngineering-Problem -> Searching with "Answer" provided the search terms I should have used (excluding the "Answer" as a search term) in my primary search for seeking the explanation I've sought :insane: :sigh: :cheers:

P.P.S.: As you said "merging before encoding" is recommended.... but simply merging the 2 parts of the source video before the encode was a NoGo for me in this case [1]... Maybe I could have edited and merged the Chapter/SRT Files by hand/script/tool to get a "synched" merge of those, properly merged the audio streams (which may, or may not have a delay (and a different one for each part))... But Simply muxing each part with Audio/Subtitles/etc. and merging the muxed parts again is IMHO so much easier compared to doing it for each kind of stream to be merged.... And that one would be another topic/thread on its own, sigh...

[1]: So far I had no concern for it, and therefore doesn't know a way to merge 2 Ifos+corresponding Vobs into a singular Source to extract from. Merging the VOBs within DGIndex or such may work fine for A/V, but is excluding (AFAIK) chapters and subtitles for that "merge"... Joining both Videos within the AviSynth Script is a viable Option, but then we have Audio/Subtitles/Chapters to be merged and synched to the video on their own...

[manono]

Originally Posted by Shandra

Merging the VOBs within DGIndex or such may work fine for A/V...

Yes, you can join the VOBs to a single D2V project file using DGIndex. If I remember correctly, at the end of the first disc there's a text screen saying something like put in the 2nd DVD. Cut that out because there's no associated audio and it'll throw off the audio for the second part. Maybe it's not included in other editions

As for the subs, extract the VOBSubs separately and join them together using the VobSub Joiner utility included in the VobSub program. Then you can convert them to SRT, although I see no real reason to do that. You can adjust their vertical position in the IDX file of the VobSubs.

That just leaves the chapter points. Depending on their format, it should mean adjusting each one from the second DVD by its frame number or time plus the final frame number or time of the first DVD. Not difficult in principle, it just takes some time.

[Shandra]

manao: Thanks for the advice, the VobSubJoiner part was new (or better to say: forgotten in time) to me (never had the need of it before, and in fact since ancient GordianKnot DivX/Xvid times I haven't touched the program itself, but just, whence needed to extract subtitles, via the MeGUI Interface for it.
As you explained the chapter part calculations -> that was exactly what I would have scripted in python (or whatsoever) for the srt subtitle parts and for the chapters (and merging end of 1 with start of 2 chapter (not sure if muxers would recognize two chapter entries for the same index)/renumbering of the 1...n of the 2nd parts chapters to n(from1)+1 to .... m)...

As you said, "it just takes some time" -> I was prepared for it, but reducing the time to just merge the two containered parts sounded better at the moment -> If I may encounter any Synch Problems for the streams from second part within the merge, it may well be to resort to the merge before encoding... Damned thing, if I really could spare my time for it -> that Python script would be completed by now (for merging srt and chapter files for a combined A/V encode, and without debugging it).

[hubblec4]

Hi Shandra

Here is another way:
Matroska Ordered Chapters!!!

Encode your DVDs separately to single mkv-files and then connect all files via Matroska-Linking. You could use Hard-Linking, but better is Medium-Linking.
All content will be played seamless.
And you can "cut"(jump over) scenes without real cutting. No Audio synchronize issues.

[Shandra]

hubblec4: Linking - As far as I understood MKV Linking, the Files won't be merged though "split" parts will be played/treaded as one by the "player" ? Any recommendations for some further readings about it Linking/Ordered Chapters (What are they and How-To) ?

[sneaker_ger]

Originally Posted by Shandra

As far as I understood MKV Linking, the Files won't be merged though "split" parts will be played/treaded as one by the "player" ?

Yes.
There are 2 ways of linking. The "simple" linking and "ordered chapters".

Originally Posted by Shandra

Any recommendations for some further readings about it Linking/Ordered Chapters (What are they and How-To) ?

http://mod16.org/hurfdurf/?p=8

[Shandra]

Originally Posted by sneaker_ger

Yes.
There are 2 ways of linking. The "simple" linking and "ordered chapters".

http://mod16.org/hurfdurf/?p=8

WOW... as long as Raspi/(Kodi-MediaCenter of choice) or similiar Solutions are supporting those specs I'll keep that in mind (I guess TV and other standalone Devices won't (If I would have aimed for those (at least those I have) SRTs wouldn't have been an option and uncropped image+vobsubs would have been a must)) - Reminds me of (Ok, the OP is going totally Offtopic here) my BladeRunner Collectors Box DVD Edition and the one DVD with 3 Movies on one Disk realized via seamless branching... So, the MKV Container could surrogate that, the missing part is somehow in the area of "how to split the original branches for the encoding"/get the indices for it and then just link each branch (ok, would mean that some parts (the major timeline) is linked via each "branch", but from what I've read so far that should be possible)...
To get back on Topic - Thanks for the Link :cheers:

[hubblec4]

Originally Posted by sneaker_ger

Yes.
There are 2 ways of linking. The "simple" linking and "ordered chapters".

The exact terms are:
1: Matroska Hard-Linking (File A+B+C)
File A knows the SegmentUID(SUID) from File B
and File B knows the SUIDs from File A and File C
and File C knows the SUID from File B.

You can start any File (A or B or C) and all content of these 3 files will be played seamless.

2: Matroska Medium-Linking
In File A(or B or C or in all 3 files) you could save Matroska Chapters with an ordered Edition(ordered chapters).
In the ChapterAtom you have to specify the ChapterSegmentUID from the files.

Originally Posted by sneaker_ger

http://mod16.org/hurfdurf/?p=8

The BEST external Matroska page I have found in the web.

Originally Posted by Shandra

WOW... as long as Raspi/(Kodi-MediaCenter of choice) or similiar Solutions are supporting those specs I'll keep that in mind...

Matroska support is unfortunately not the best atm.
All software players which can use LAV Filters are the best.
Hard- and Medium-Linking is well supported.

Originally Posted by Shandra

my BladeRunner Collectors Box DVD Edition and the one DVD with 3 Movies on one Disk realized via seamless branching... So, the MKV Container could surrogate that, the missing part is somehow in the area of "how to split the original branches for the encoding"/get the indices for it and then just link each branch (ok, would mean that some parts (the major timeline) is linked via each "branch", but from what I've read so far that should be possible)...

Multi-Edition DVDs are not as widespread, and it is also a complicated topic.
With chapterEditor(1.00) is it possible to generate a Multi-Edition mkv from such Multi-Edition DVD's.