chapterEditor(Chapter-Database Disc2mkv Matroska-Tools[Tags,Menu,Multi-MKV])+CLI

[hubblec4]

Hi all

I can't login to the chapter database and I get an error message that the security token is invalid. Can someone confirm this?

I have changed nothing so I guess the host/server has changed something.

[gonca]

I get the same result

[hubblec4]

Thanks for your feedback.
Indeed the server has changed something for the Cookies.

Here is a test version for Windows.

[hubblec4]

chapterEditor(cE) rev1.38 is available.
Windows
Linux


A hotfix for the chapter database because the server changed something and you could no longer log in.

Change chapterEditor

added    &
fixed    *
improved #
changed  %


rev1.38 2023-06-08
// chapterEditor
* Font color and the color for the settings button in the sidebar are now set correctly

* Chapter files(.txt) with the time format "HH:MM:SS:FF" are now recognized correctly again

# H:MM:SS.MS/NS time formats are now supported


// Chapter Database
* Changes made by the server have been adjusted
  - older cE versions no longer have access


// DVD2mkv
* forced subtitles that were created automatically can now be edited

[RivDoVi]

Hullo there and thanks for this incredible program !

I'm trying to use the Multi-Edition-MKV mode of BD2mkv to create 2in1 editions of Alien on UHD as well as Aliens/3/Resurrection to get both Special cut & Theatrical.

However, after several hours trying different things and combination, i cannot generate an mkv that plays the second edition of Alien UHD flawlessly.
It seems that, when transitioning between parts, i'm having flashes of other parts for a split second, if that makes sense, or a small freeze/hiccup.
After a close look at the second edition generated chapters, particularly on the hidden chapters, it seems their timestamps are wrong by a few ms or 1 second, which seems to explain why i see another scene for a split second.
No problem on the first edition, problem only occurs on second one.

I tried extracting audio tracks with eac3to, no change.

I tried manual muxing after chaptereditor job was done as well, and i get the following error :
Only partial append mappings were given for the file no. 1 ('BDMV\STREAM\00703.m2ts'). Either don't specify any mapping (in which case the default mapping will be used) or specify a mapping for all tracks that are to be copied.

Is this a known problem ?
If someone has an idea, that would be very appreciated. I've been trying my best to solve it but didn't find a way after several hours of tries ....

Here is the mediainfo of the generated mkv if it can be of any use : https://pastebin.com/a13G836S

Thank you kindly

[hubblec4]

Hi RivDoVi
and welcome to Doom9 forum.

Originally Posted by RivDoVi

I'm trying to use the Multi-Edition-MKV mode of BD2mkv to create 2in1 editions of Alien on UHD as well as Aliens/3/Resurrection to get both Special cut & Theatrical.

For creating the BD2mkv editor I have used the Alien movies for testing, but it was Blu-ray not UHD.

Originally Posted by RivDoVi

However, after several hours trying different things and combination, i cannot generate an mkv that plays the second edition of Alien UHD flawlessly.
It seems that, when transitioning between parts, i'm having flashes of other parts for a split second, if that makes sense, or a small freeze/hiccup.
...

I tried extracting audio tracks with eac3to, no change.

The problem you describe is related to the audio tracks because there are duplicate frames.
All Audio streams which are used in the mkv have to be demuxed with eac3to.
OK you said you had used eac3to to demux Audio streams. Have you all Audios demuxed?
I'm also not sure if ea3cto handles UHDs corretly. That's really a problem because eac3to is no longer maintained.
Currently I work (a bit) on the implement of DGDemux. I had asked for some improvements which are now included.
DGDemux is up to date and has nice features.

Originally Posted by RivDoVi

After a close look at the second edition generated chapters, particularly on the hidden chapters, it seems their timestamps are wrong by a few ms or 1 second, which seems to explain why i see another scene for a split second.
No problem on the first edition, problem only occurs on second one.

The first edition is always straight forward because the m2ts files in the correct order.
For the second edition (and all others) the m2ts files not in the correct order, because these m2ts files are appended after the first editions m2ts files.

The result of the Audio duplicated frames issue is, that the chapter markers no longer match to the correct frame.
The more m2ts files are available, the further the chapter markers are seemingly shifted.
The chapter times should be OK, but maybe not if there are problems with creating.

Originally Posted by RivDoVi

I tried manual muxing after chaptereditor job was done as well, and i get the following error :
Only partial append mappings were given for the file no. 1 ('BDMV\STREAM\00703.m2ts'). Either don't specify any mapping (in which case the default mapping will be used) or specify a mapping for all tracks that are to be copied.

Is this a known problem ?

OK, that's interesting. Can you check the duration of the 00703.m2ts file.
I guess it is a short one and such "bad" first m2ts files a removed by cE, normally automatically.
But sometimes the duration a bit longer as the default value (2 seconds). You can change this in the BD2mkv settings.

Originally Posted by RivDoVi

Here is the mediainfo of the generated mkv if it can be of any use : https://pastebin.com/a13G836S

More helpful for me would be:
1) generated files by cE -> chapters.xml, tags.xml and so on, but not the demuxed streams or other big files.
2) a Disc chunk created with my DiscShare tool (to find in my signature)
- use minimal the "-f" switch this will create a chunk with all the meta data files only
- or use "-f" and select some m2ts numbers to include a chunk of them. for example the 00703.m2ts

[RivDoVi]

Originally Posted by hubblec4

Hi RivDoVi
and welcome to Doom9 forum.
[...]

Thanks a lot for your answer !
Yes i only encounter this problem with the UHD movie ; I tried Alien 3 BD and there was no problem.
First, i had to use eac3to 3.34 instead of 3.36 for the UHD as eac3to 3.36 was incapable to extract the hevc/h265 from m2ts ; i saw it was a known issue in the related doom thread.
I also took a look at the demux log and you are right about the audio : i have dozens and dozens of :
[a17] Audio has a gap of 17ms at playtime 2:38:49. <WARNING>
Yes, i demuxed all streams.
When demuxing Alien 3 BD (not uhd) i had no such warning in the log file. It seems all m2ts of Alien UHD (or maybe all seamless branching UHD ?) have this gap for m2ts ?

Here is the log : https://pastebin.com/6gAPeV2S
Also, here are the tags + Chapter xml requested : https://pastebin.com/EWdk65yh

It seems the problem is clearly eac3to related ; the disk is perfectly fine as i extracted and watched both cut separately without any problem.

I also tried no demuxing of files & using tsMuxer instead of eac3to and the chapters were still off.

I checked the 00703.m2ts duration and it's 40 seconds ; i also checked all m2ts duration and the shortest is 31 sec so i'm sure it's not the problem here .

I also had the idea to manually append all the m2ts files in the correct order, not using the mtxcfg, with mkvtoolnix and just putting the tags & chapters.xml generated by CE and they play flawlessly that way, no freeze or hiccup, audio has no drop out at all because all m2ts files have an I-frame at the start and the 2 editions are recognized, however the chapters generated by cE for the second edition are still off by a second or a few ms so the second edition is still off (though no freeze or hiccup as i said).

Yes i also use DGDemux but i found no way to use it with cE. If support was implemented that could really be useful as they have proper UHD & Dolby Vision support ; i'm also 90% sure it would solve the gap issue.

[hubblec4]

Originally Posted by RivDoVi

Yes i only encounter this problem with the UHD movie ; I tried Alien 3 BD and there was no problem.

Ok very interesting and good to know that Blu-ray works and UHD not. Could be a hint that eac3to is the bottleneck/issue.

Originally Posted by RivDoVi

....
I also took a look at the demux log and you are right about the audio : i have dozens and dozens of :
[a17] Audio has a gap of 17ms at playtime 2:38:49. <WARNING>
Yes, i demuxed all streams.
When demuxing Alien 3 BD (not uhd) i had no such warning in the log file. It seems all m2ts of Alien UHD (or maybe all seamless branching UHD ?) have this gap for m2ts ?

Yes all movies which are splitted to multiple m2ts files have such gaps(99,5%).

Originally Posted by RivDoVi

It seems the problem is clearly eac3to related ; the disk is perfectly fine as i extracted and watched both cut separately without any problem.

I also tried no demuxing of files & using tsMuxer instead of eac3to and the chapters were still off.

Unfortunately MKVToolNix(MTX) is not able to remove this gaps and we have to do this with eac3to, because it is the only tool which can import a m2ts list(not only mpls files) and can remove the gaps.
TsMuxer accepts also a m2ts list but can not remove the gaps.
DGDemux can remove gaps but has no import support for a m2ts list. I have asked for that feature but currently is there no progress.
DGDecNV has both and I will test it when I have time. But DGDecNV is not free to use, you need a license and a NVIDIA graphic card.

Originally Posted by RivDoVi

I checked the 00703.m2ts duration and it's 40 seconds ; i also checked all m2ts duration and the shortest is 31 sec so i'm sure it's not the problem here .

OK. But the playing time is not really the decisive criterion, you would have to take a closer look at the streams that are included.
The playing time was only a good indicator to roughly recognize whether it is a bad m2ts file.

Originally Posted by RivDoVi

I also had the idea to manually append all the m2ts files in the correct order, not using the mtxcfg, with mkvtoolnix and just putting the tags & chapters.xml generated by CE and they play flawlessly that way, no freeze or hiccup, audio has no drop out at all because all m2ts files have an I-frame at the start and the 2 editions are recognized, however the chapters generated by cE for the second edition are still off by a second or a few ms so the second edition is still off (though no freeze or hiccup as i said).

A lot of work for manually input :-)
And like I said, MTX is not able to remove the Audio gaps which ends up with the wrong video frame for the chapter markers.
In that case when all playing fine you could manually shift the chapter times, but that's much much work and you have to life with not correct audio streams.

Originally Posted by RivDoVi

Yes i also use DGDemux but i found no way to use it with cE. If support was implemented that could really be useful as they have proper UHD & Dolby Vision support ; i'm also 90% sure it would solve the gap issue.

Yes that's the plan, to make cE like Makemkv with DGDemux, because all can be muxed with MTX(the author of DGDemux promise it).

[RivDoVi]

Originally Posted by hubblec4

Ok very interesting and good to know that Blu-ray works and UHD not. Could be a hint that eac3to is the bottleneck/issue.

Okay I managed to totally FIX the problem with DGDemux, it was a LOT of work but HURRAY it works :
I demuxed the first edition 800.mpls (Theatrical Cut) ;
Then i demuxed the second edition 801.mpls (Director's Cut) but checked "episode demuxing" to get separate m2ts demuxing.
I then manually picked and appended all the required files of the DC to the main files from 800 mpls above using MKVToolnix.
In the same mkvtoolnix window, i dropped the chapters.xml & tags.xml generated by cE, muxed and VOILA, no issue anymore, perfect sync, perfect transition, perfect chapters timestamps !

So yes, you got your answer : DGDemux is the solution to all the problems !
Would be very great if you could consider adding it as this example is the proof that it resolves all the issues encountered and DGDemux would add support for Dolby Vision as well.

I thank you for your time, i learned new things and hope i was able to give you useful informations as well
Though it now works, it really is a lot of work without DGDemux support lol.

[hubblec4]

Wow, that are really great news, especially that it works with the "episode demuxing".
I thought that appending of a lot of small streams will end up again with other issues.

If I understand this correctly you get a lot of demuxed files: for each m2ts there are all the streams you want to preserve.
And all files from all the m2ts files you have appended manually in MTX?

I could use this maybe as a workaround for DGDemux for the Mutli-Edition creation.

An other workaround which is in my mind is to generate a new mpls file with all the required m2ts files.
And yes I have also learned from you.

[RivDoVi]

Here are the steps :
- Demuxing 00800.mpls normally ; no "episode demuxing" here as it would be pointless since this is the first edition.
- Demuxing 00801.mpls, checking episode demuxing this time to be able to use only the required m2ts parts once extracted..
This gives indeed separate extracted video/audio/subs files for all parts.
- Since all parts are labelled [1] [2] etc (example : 5900 [1] PID 1011.hevc) ... i removed the unneeded ones using the "M2TS list" provided by DGDemux, for alien it was 1/2 to delete which is quite logical so i only kept the parts with even numbers (2/4/6 etc)
- I then appended all these parts to the corresponding audio/video/subs etc... part from 00800.mpls in mkvtoolnix
- I then dropped the tags.xml and the chapters.xml that i previously generated with BD2mkv cE to mkvtoolnix and then muxed everything.

It now works absolutely flawlessly.

I can't vouch for very very short m2ts file like 1/2s as my shortest was indeed 30 second (real playtime) but if you have an example i could try it for you

Also a quick note : I tried "chop excess audio" during my first trial with DGDemux and this screwed audio in the last portion of the mkv so avoid to check that lol.

Of course, creating a custom playlist would avoid all these steps as you say. This would be the best option but i have absolutely no knowledge as to how to create a playlist sadly.

Hope it's useful !

[hubblec4]

Thanks for your answer.
The steps are good described and I know what is to do.

I have also asked before a few seconds in the DGDemux forum to an option for the "-ep" demux to specify a list of needed parts/m2ts files.
https://www.rationalqm.us/board/view...p=17262#p17262

I think a small m2ts file with a short duration should also work.
For -ep demuxing DGDemux don't accept the -sf and -sl switches but this can be ignored and cE will not use such short m2ts files.

OK, good to know that "chop excess audio" not work. I don't understand fully what are the differences between this and remove gaps and strict-playlist.

I understand well the mpls structure but it is different to create a new one, so this will be my last option because I think this needs some time to learn.

[RivDoVi]

Okay browsing the forum you linked i just stumbled upon this thread : https://www.rationalqm.us/board/view...hp?f=16&t=1174
Second post talks about DGIndexNV being capable of extracting separate M2TS.
I then tried and it worked ! I loaded the playlist 00801.mpls in DGIndexNV, then went to "open" and deleted the unnecessary m2ts files and reorganised the files as i wanted them to be, selected the audio/subs i wanted to be kept, and then clicked "save project".
It extracted and joined all M2TS selected in the order specified.
While it doesn't extract all M2TS separately, this is very useful because i obtained all the requested second edition parts in 1 file for each audio/video/subs.

So in theory it should be possible to use DGDemux to extract the main playlist, then use DGIndexNV for the parts needed for second,third etc... editions.
I don't know if DGIndexNV support Dolby Vision though as this is mandatory to correctly extract high quality UHD like Gladiator. Losing Dolby Vision is not conceivable.

This is still not a fully automated process but it makes the task less cumbersome as extracting every m2ts with DGDemux was really a pain !

[hubblec4]

Many thanks for this tests, I will report this to DGDemux because he has think about to implement this DGIndexNV(sorry I has said before DGDecNV) feature into DGDemux.
Could you make another test with DGIndexNV where you load the 00800.mpls and add all other missing m2ts from 00801.mpls to get one fully stream(per stream type) for all Editions?

When this feature is implemented in DGDemux all will be very easy.

I have already a simple support for DGIndexNV in cE it is used in the BD2mkv editor for the Split feature because I can load a m2ts list.
But this tool is not free. So I want implement it for demuxing in BD2mkv.

[hubblec4]

A note: You don't need to extract others then audio streams. Only this stream type is the bottleneck. Video and subtitle streams are well handled by MKVToolNix.
But for DoVi is a demux for the video necessary.

This should safe time for the moment for manually creating.

[RivDoVi]

Hi.
Okay i was able to load the 00800.mpls, then i manually added the required 00801 parts at the end and it seemed to work as well. However it's a bit cumbersome to do in DGIndexNV because when i added multiple m2ts they were sorted by title. Which means if your second edition has files named 687/688 and your first editions are labeled 700/701 etc... the second edition parts will be on top, so you have to manually descend each file one after another lol. But yes it worked during my test and is a good temporal solution.
However, since the objective is to be able to correctly support UHD discs, there is no DV support at all in DGIndex, i just tried with Gladiator and the DV layer wasn't processed so yes, DGDemux support would be a lot better.

Also, thanks for the tip. I knew MKVToolnix could correctly cut video files but i wasn't sure about the subs. Thanks a lot, that will help me save my SSD a bit, i think i must have processed a few TB of data for all these tests lol .

[hubblec4]

Many many thanks for your testing and I'm very happy to read it works.

I have also requested for DGDemux to implement the m2ts-list loading feature from DGIndexNV and your test proves it works.
I have made a bit progress last night to implement DGDemux in cE but I think I will take some time more to be finished.

Currently I have no UHD disc with DoVi for testing.
Gladiator is such a disc, do you know other discs with DoVi?

[RivDoVi]

I'll be glad to test it once it's done, take your time ! I'm happy to be able to help as i"m a remuxing nerd as well lol.

You can find pretty much all Dolby Vision bluray titles here : https://www.nextgenhometheater.com/d...lu-ray-movies/

I took the example of gladiator because it has both Dolby Vision, seamless branching & 2 cuts of the movie on the disc which makes it a perfect guinea pig !

Edit : Okay i also tried a different type of "custom 2in1" for the sake of experiment : I took an extended cut dolby vision remux of Gladiator in .mkv and tried to mux the theatrical cut part in m2ts format from the UHD Bluray. I used MakeMKV to extract the video parts to retain the DV layer for each part and DGIndexNV for subs & audio.
To my greatest surprise, cE was able to create the chapters required for both cut while just looking at the mpls files, i did not even have to copy all m2ts, it seems the mpls & clipinf files are enough to generate chapters in cE which is very good !
So the good news is that i was able to correctly create a new Dolby Vision mkv with both cut, synced audio, but sadly there is a black frame for a split second before each theatrical cut parts transition, so yeah, gonna have to stick to full m2ts when creating 2in1 editions here as muxing mkv+m2ts source files doesn't do well i suppose ^^ !

[hubblec4]

Thanks for the link.
Gladiator seems indeed a good choice for testing, it is not my favorite movie but I will order it,maybe I find another one :-)

cE needs only the mpls files to create the magic Xin1 chapters and this shows me how powerful Matroska chapters can be, if the player has a proper support.
Blu-ray uses more or less the same system like ordered chapters. For each m2ts which is used there are IN- and OUT- times -> chapter start - chapter end.

Interesting test, but not a perfect solution and yes, wait a bit. I don't know if Makemkv removes the Audio gaps correctly.
Also the m2ts files of the first Edition need to be treated.
I hope DGDemux gets the m2ts-list import feature, if not I will code the workaround with the -ep demux option.

[hubblec4]

Here is a first test version with the basic support for DGDemux.

DoVi and multi-Edition-mkv are currently not yet implemented.