About "Skipping Identical Frames" on Eac3to?

[theliver9x]

Sometimes I get the message "Skipping identical DTS or AC3 frames (seamless branching).
That means it removed some frames from the original disc. So does it cause loss of original movie data and I am very worried that the software will cause the movie to lose some images making it not full content like the original movie that the producer made.
Can anyone explain this to me. Thanks.

[filler56789]

Audio frames are groups of digital-audio samples.

(Yes, the tech terminology sucks, the "pros" love to be unimaginative and prefer adding new meanings to old words -_-)

Therefore, no, eac3to doesn't remove images from the video streams.
BUT the removal of audio frames MAY cause loss of synchronization,
generally speaking,
but I admit that I have no experience with seamless branching :-/

[Columbo]

BUT the removal of audio frames MAY cause loss of synchronization, generally speaking :-/ No, the point of removing the audio frames is to maintain sync for seamless branching.

[filler56789]

No, the point of removing the audio frames is to maintain sync for seamless branching.

Thanks :rolleyes: for erasing the part that says that I might be wrong.

[tebasuna51]

No, the point of removing the audio frames is to maintain sync for seamless branching.

This is how multiple M2TS files are generated: the cut is exact in video frames, but it cannot be exact in audio frames at the same time.

Therefore, audio frames are repeated at the end of one M2TS file and the beginning of the next, compensated for by timestamps, and these must be removed to generate complete audio while maintaining synchronization.

[Columbo]

Yes, exactly what I said.

[tebasuna51]

Yes, exactly what I said.

Of course, I just wanted to explain the reason.:goodpost:

[Columbo]

Got it, thank you.

[hubblec4]

Everything written so far is not entirely correct.

There are no duplicate audio frames on Blu-rays!

The problem is MkVToolnix when muxing; there's a serious bug.

The workaround that eac3to and DGDemux use is OK, but it removes original audio frames, which ALWAYS leads to a slight synchronization issue.

[Columbo]

Strongs claims need strong evidence. May I ask for your evidence?

[hubblec4]

Hi

First the bug
https://codeberg.org/mbunkus/mkvtoolnix/issues/3728

My chapterEditor has a fix for that MKVToolNix bug
https://forum.videohelp.com/threads/368560-chapterEditor(Chapter-Database-Disc2mkv-Matroska-Tools[Tags-Menu-Multi-MKV]/page6#post2754681

Unfortunately, I can't quickly find the relevant section in the Blu-ray specifications I have here.

But to summarize: Each play item must begin with a video keyframe, and the audio frame must not have an offset.

Finally, each play item may start an audio frame, as long as it's within the last video frame.

Inevitably, the last audio frame will "run" a tiny bit longer than the last video frame.

But no new audio frames may start after the last video frame.

And the problem is this: MKVToolNix appends the first video frame of the second play item directly after the timestamp of the last audio frame of the first play item. This creates a video gap of a few milliseconds. This quickly adds up when there are many play items.

[Columbo]

Don't fully follow you, but are you claiming no gaps correction is needed, or just that the issue is not audio frame duplication? domy has pretty clearly shown frame duplication without MKV being involved.

DG's gaps correction doesn't rely on duplication. It just accumulates elapsed time for audio and video and compares them. Kind of like Bresenham's algorithm.

Sounds like your MKV issue is a different thing than M2TS gaps correction not involving MKV.

[hubblec4]

Eac3to and DGDemux work like this:
Audio and video times are continuously added together, and when the audio becomes approximately one frame longer than the video, an audio frame is simply discarded.

Yes, my chapterEditor (cE) doesn't perform any gap correction because it's not necessary.

All cE does is create a timestamp file for audio frames, which then tells MKVToolNix how to handle them. Each new play item is then appended directly to the previous one, seamlessly, for both video and audio.

[Columbo]

Yes, I know how eac3to and DG do correction. I am a DG Tools team member.

So this is exclusively an MKV issue? That's all I'm asking because your post claimed that there is never any duplicated audio frames but that is demonstraby false. It arises when demuxing ES. Read the spec about connection conditions.

I think preparing things for MKV muxing is different from demuxing ES from multiple seamless branched M2TS's.

[hubblec4]

It's definitely a problem for MKV, yes. But I'm not familiar with demuxing ES from m2ts files.

However, I've always searched the MKV files for "duplicate" audio frames, and by duplicate, I mean bit-identical.

In any case, MKVToolNix shows an Adler checksum for all frames, and this should be identical if there are duplicate audio frames. But I haven't found any identical Adler checksums at the transitions between one m2ts file and the next.

[Columbo]

Gotcha, thanks for the clarification.

[tebasuna51]

It's definitely a problem for MKV, yes. But I'm not familiar with demuxing ES from m2ts files.

However, I've always searched the MKV files for "duplicate" audio frames, and by duplicate, I mean bit-identical.

In any case, MKVToolNix shows an Adler checksum for all frames, and this should be identical if there are duplicate audio frames. But I haven't found any identical Adler checksums at the transitions between one m2ts file and the next.

That may be, but searching in mkv's cannot prove that m2ts files (Blu-Ray) do not have duplicate audio frames:

There are no duplicate audio frames on Blu-rays!

I extracted audio tracks from two adjacent m2ts (seamless branching) and they are bit-identical frames (last and first).

[hubblec4]

That may be, but searching in mkv's cannot prove that m2ts files (Blu-Ray) do not have duplicate audio frames:

Let's put it this way: for me, what's in the final MKV file is sufficient, since I completely trust MKVToolNix(MTX).

Mosu never programmed in gap correction or the removal of audio frames.

Every audio frame is used, and if no bit-identical frames are found, then there were no identical frames for this Blu-ray.




I extracted audio tracks from two adjacent m2ts (seamless branching) and they are bit-identical frames (last and first).

Could you please provide the name of the Blu-ray.
I'm very sure it is only one of 10000 BDs which have identical frames.

[Columbo]

MONSTERS_UNIVERSITY

See the FAQ here for some pretty pictures.

https://github.com/domyd/mlp

[tebasuna51]

The mlp demux is a exact method to extract even thd tracks (with frames 1/1200 ms), but with AC3 frames (32 ms) was more necesary like I show in 2012 (https://forum.doom9.org/showthread.php?p=1600694#post1600694). All seamless branching BD's can be a example.

[Columbo]

but with AC3 frames (32 ms) was more necessary Exactly! And that's why DG stuff uses the "Bresenham's approach", it works with all formats.

[hubblec4]

Many thanks for this link and info.

Yes, for TrueHD is it more complex. The reason is the very short duration for TrueHD frames (1/1200).
One frame alone is it really noticeable (a real question, no snark)?
I don't know how many TrueHD frames are needed to be noticeable.

In principle, the Blu-ray creators could have simply omitted the last few frames that follow the final video frame.
But I suspect they consider several TrueHD frames together as a single "meaningful frame."

That's why there are so many TrueHD frames.
Okay, so we can say: there can be identical frames for TrueHD streams.


I'll order the Blu-ray with my next movie order, because this looks very interesting.

[Columbo]

CARS_2 has 94 segments IIRC, or was it MONSTERS.

1/1200 * 94 = ~78ms

People would not be wanting gaps correction with THD if it were not an issue.

Even without identical frames there are issues, because each segment does not end cleanly with end of video and end of audio coinciding. As tebasuna pointed out, with a 32ms frame size, this quickly accumulates. The DG approach handles this with a single algorithm for all formats. The only difference is what you can cut without breaking the stream.

For a player it doesn't matter, because the PTS timestamps can be used to qualify the playback across the seam. But when demuxing ES the timestamps are lost.

[hubblec4]

CARS_2 has 94 segments IIRC, or was it MONSTERS.

1/1200 * 94 = ~78ms

OK. Interesting, 78ms are very long in relation to other audio frames.


People would not be wanting gaps correction with THD if it were not an issue.
Yes. There are issues and we need solutions. One for demuxing, and one for direct stream copy,write to MKV.


Even without identical frames there are issues, because each segment does not end cleanly with end of video and end of audio coinciding. As tebasuna pointed out, with a 32ms frame size, this quickly accumulates. The DG approach handles this with a single algorithm for all formats. The only difference is what you can cut without breaking the stream.


Yes, that's true; whenever a film consists of multiple segments, there are problems with the transitions.

But this problem can be directly solved when creating an MKV file; or rather, it doesn't occur in the first place because the frames, audio, and video always retain their exact timestamps.


For a player it doesn't matter, because the PTS timestamps can be used to qualify the playback across the seam. But when demuxing ES the timestamps are lost.
Yes, exactly, that's the difference. Missing timestamps are always a problem, of course.

-----------------
A general question about the algorithm for removing audio frames.

Since you're familiar with this, could you perhaps confirm the following or correct me if necessary?

At the end of each M2TS file, the algorithm surely checks how much excess audio has accumulated.

If it reaches (or almost reaches) the duration of a video frame, an audio frame is discarded. I'd like to know which audio frame is discarded here: the one from the current M2TS file or the next one?

After the audio frame is removed, it's still very unlikely that the audio and video will be exactly the same length. A small offset of 1ms to 10ms is certainly not uncommon.

Let's say that an audio frame can be removed after the second M2TS file, and we cut the last audio frame from the current M2TS file.
However, an offset of 4ms remains.

This means that the first audio frame of the third m2ts file is no longer synchronized with the first video frame of the third m2ts file. All audio frames from the third m2ts file now have a 4ms offset to the video frames.

Is this correct?

[Columbo]

The accumulated video and audio times are maintained. When the audio excess passes the duration of 1/2 an audio frame, the audio stream is marked for needing a deletion. That causes the next audio frame to be deleted.

The cut will have made things such that the current state is that accumulated vid duration > audio duration, and the algorithm won't get triggered again until enough excess audio has again accumulated. Just like the Bresenham line drawing algorithm hugging the true line, the gaps algorithm hugs a desync of ~1/2 audio frame duration. It does not necessarily delete a frame at every gap but rather only when needed.

In practice pathologies occur that make the audio excess larger than 1 frame so the algorithm allows for multiple frames to be deleted if needed.

Of course, it's not "perfect" but it works well in practice. And it is applicable to all formats.

domy's idea was to try to make things perfect. But it can't be perfect for various reasons, and if you try to salvage the idea, edge cases proliferate and make things quite complex. For example, audio content can be content "identical" but not bit identical (think of low-order bit differences). So now to detect duplicated content you need a fuzzy comparison of some kind, negating to some extent the idea of perfection. You can see by looking at the issues at his git (https://github.com/domyd/mlp/issues) that he abandoned trying to keep things working in the face of edge cases and pathologies, and stopped development. DG's algorithm is simple and robust and works well enough to be quite useful. We've been maintaining our tools since 2003.

Improvements have been suggested and we're happy to consider them but so far nothing has been proposed that justifies the added complexity. One criticism that has been leveled is that if the video exceeds the audio at gaps then correction is never triggered, but for bluray we have never seen that in practice and it may be counter to the spec in any case.

Now you have all our secrets! Watch me get fired.

I understand that your method for MKV in effect retains the timestamps for things, which in conjunction with some mkvtoolnix magic and expected behavior of the players allows you to bypass the gaps problem. That is a creative approach, but doesn't work for demuxing ES, where timestamps are lost. You acknowledged that when you said:

"There are issues and we need solutions. One for demuxing, and one for direct stream copy, write to MKV."

I note that MKV is not the only target format, and that ES is often needed for other things than immediate playback.

[hubblec4]

The accumulated video and audio times are maintained. When the audio excess passes the duration of 1/2 an audio frame, the audio stream is marked for needing a deletion. That causes the next audio frame to be deleted.
.......
Of course, it's not "perfect" but it works well in practice. And it is applicable to all formats.

Thank you very much for the explanations.
I have one small comment about this.
The Blu-ray specs state that a new m2ts file must always begin with a video and audio keyframe.
Therefore, I'm wondering if it wouldn't be better NOT to delete the next audio frame, because it definitely/usually/should be a keyframe.


domy's idea was to try to make things perfect. But it can't be perfect for various reasons, and if you try to salvage the idea, edge cases proliferate and make things quite complex. For example, audio content can be content "identical" but not bit identical (think of low-order bit differences). So now to detect duplicated content you need a fuzzy comparison of some kind, negating to some extent the idea of perfection. You can see by looking at the issues at his git (https://github.com/domyd/mlp/issues) that he abandoned trying to keep things working in the face of edge cases and pathologies, and stopped development. DG's algorithm is simple and robust and works well enough to be quite useful. We've been maintaining our tools since 2003.

Yes, perfection is always a tricky thing. One can invest a lot of time and energy and still not achieve success.
And I've also been dealing with this problem since 2002. It took me a lot of time to understand why the error occurs and where in the processing chain something is wrong.
I never would have thought it was MTX that wasn't working optimally. It's not an MTX fault, but Mosu made a decision back then about how streams are appended. There's not much that can be done about that now.
The DG-Team is so great!!! I'm happy that DGDemux exists!




Improvements have been suggested and we're happy to consider them but so far nothing has been proposed that justifies the added complexity. One criticism that has been leveled is that if the video exceeds the audio at gaps then correction is never triggered, but for bluray we have never seen that in practice and it may be counter to the spec in any case.

Yes, I can well believe that; many will have encountered this problem and considered it.
One possibility would be to create a timestamp file during demuxing. This could then be reused whenever the audio stream needs to be processed.


Now you have all our secrets! Watch me get fired.
I don't think Rocky will fire you, he's a really nice guy.



I understand that your method for MKV in effect retains the timestamps for things, which in conjunction with some mkvtoolnix magic and expected behavior of the players allows you to bypass the gaps problem. That is a creative approach, but doesn't work for demuxing ES, where timestamps are lost. ...
First of all: MKVToolNix (MTX) doesn't perform any magic here; on the contrary, MTX needs help so it doesn't automatically decide how the timestamps should look. Without a timestamp file, MTX won't correctly join the tracks.

Similarly, the players themselves don't really have any magic. VLC, for example, can't handle video gaps at all; the movie always freezes briefly, and video artifacts appear. The best players here are MPC-HC or mpv.
The whole magic is just the timestamp file.


I note that MKV is not the only target format, and that ES is often needed for other things than immediate playback.
You're absolutely right. I sometimes forget that there are other containers besides Matroska. For me, there's only Matroska, as it's the best container in the world.

[Columbo]

Thank you for your reply.

The Blu-ray specs state that a new m2ts file must always begin with a video and audio keyframe.
Therefore, I'm wondering if it wouldn't be better NOT to delete the next audio frame, because it definitely/usually/should be a keyframe. The only relevant audio format for which every frame is not a keyframe is THD. That is handled differently as I implied when I said that the only difference between formats is what you are allowed to cut. Also, if the two frames are duplicates it doesn't matter which one you cut.

The DG-Team is so great!!! I'm happy that DGDemux exists! Thanks, I appreciate that.

I don't think Rocky will fire you, he's a really nice guy. Yes he is.

First of all: MKVToolNix (MTX) doesn't perform any magic here; on the contrary, MTX needs help so it doesn't automatically decide how the timestamps should look. Without a timestamp file, MTX won't correctly join the tracks. The magic is that it accepts a timestamps file.

Similarly, the players themselves don't really have any magic. VLC, for example, can't handle video gaps at all; the movie always freezes briefly, and video artifacts appear. The best players here are MPC-HC or mpv. The whole magic is just the timestamp file. I was thinking of your multi-edition support. Does that not require player support?


You're absolutely right. I sometimes forget that there are other containers besides Matroska. For me, there's only Matroska, as it's the best container in the world. When all you have is a hammer, everything is a nail. ;)

[hubblec4]

Thank you for your reply.

The only relevant audio format for which every frame is not a keyframe is THD. That is handled differently as I implied when I said that the only difference between formats is what you are allowed to cut. Also, if the two frames are duplicates it doesn't matter which one you cut.
Okay. I understand that TrueHD is really different. I'm really looking forward to the Blu-ray when I order it.



I was thinking of your multi-edition support. Does that not require player support?


Yes, a player needs to be able to do certain things to provide proper support for multi-editions.
Ordered chapters are already built into many players, not always well or optimally, but at least the basics are there.
MPC-HC does all of this very well, and mpv is close behind. If you want to test some more Matroska features, you can try my mpvMatroska.

[Columbo]

Can I use your timestamps method to make a normal single edition MKV from files that have not been gaps-corrected during demux and that will be in sync? What would the process be?

[hubblec4]

Hi Columbo

Absolute yes. A single Edition is the same as multiple Editions. The different is only: the used m2ts files.
But my cE not works with a single Edition in the Multi-Edition mode.

To create the timestamp files manually is not really easy(almost impossible).
In short:
1. the main timeline are the video frames
2. count the audio frames in each m2ts
3. the first audio frame of each m2ts gets the timestamp of the first video frame from the m2ts.

[Columbo]

To create the timestamp files manually is not really easy (almost impossible).
That's a deal breaker. Don't you have a tool that will do this for single edition? I guess it would be quite popular.

[hubblec4]

I don't think so. But I don't know enough about it.

I asked Rocky back then if it would make sense for DGDemux to also create timestamp files while extracting the tracks. But we decided against it.

[Columbo]

Why did you decide against it?

[hubblec4]

Why did you decide against it?
To complicated, to less interested user.

I see my cE is able to create Timestamp files for single Editions, but it wont work for THD. I use a "fast" mode for TS-file creation, and THD breaks this small algo.
As info: THD has two FPS values: 1/1200 for 48kHz; and 1/1102,5 for 44,1kHz

I have no time currently to work on cE to fix that. But maybe future.

[Balling]

The only one who handled it correctly is MakeMKV after 1.15.4 i was present during this all. https://forum.makemkv.com/forum/viewtopic.php?t=24048. It actually preserved first frame of next m2ts, which is a key frame (yes, TrueHD has key frames, ffmpeg had to implement key frames for mp4 container for TrueHD) and zeroed out the last subframe of previous segment. Also it is strange to say https://github.com/domyd/mlp/issues stopped development. He just archived the repo. That does not mean anything, the code is complete. There are no bugs there. It does almost the same thing as MakeMKV 1.15.4.

[Columbo]

The only one who handled it correctly is MakeMKV after 1.15.4 That is false. The DG tools use the same approach, as does mlp, and in fact, as domy acknowledges, DG was the first to discover the correct way to cut frames from THD, and thereby create a solution to the sync issue. BTW, DG also created the first tool to merge THD and AC3 streams.

Also it is strange to say https://github.com/domyd/mlp/issues stopped development. He just archived the repo. The last commit was 4 years ago so it's not strange at all.

That does not mean anything, the code is complete. There are no bugs there. There are 5 open issues, two of which are feature requests and 3 real issues. If you choose to not call the issues 'bugs' so you can ignore them, that's just word games.

Finally, I remind you that making MKVs is not the only use case, that THD is not the only audio format, and that DG was the first to solve sync issues with seamless branching when demuxing.

[Balling]

"There are 5 open issues, two of which are feature requests and 3 real issues. If you choose to not call the issues 'bugs' so you can ignore them, that's just word games."

Those bugs are caused by more complex timestamps in mpeg-ts container. If you extract the thd files with ffmpeg first the issue is no longer there. See this: https://rationalqm.us/board/viewtopic.php?t=1376&sid=daab66cfea258297e3b6aa978acac80a

This feature request is a bad idea https://github.com/domyd/mlp/issues/12 because interleaved THD+AC3 files cannot be played by ffmpeg, it was specifically considered unacceptable format by ffmpeg. But to be fair ffmpeg is not a good m2ts blu-ray writer anyways.

"I've read, DGDemux only removes the last minor from a previous segment if the next segment starts with 2 major sync frames"

Oh, nice, this is what MakeMKV does. There is still an issue with ffmpeg warning that lossless check failed, right?

[Columbo]

Those bugs are caused by more complex timestamps in mpeg-ts container. Glad you acknowledge the bugs now. That is one possible mechanism but it doesn't cover all the cases.

This feature request is a bad idea https://github.com/domyd/mlp/issues/12 because interleaved THD+AC3 files cannot be played by ffmpeg, it was specifically considered unacceptable format by ffmpeg. It's not just about ffmpeg. Bluray requires THD tracks to have an embedded AC3 fallback. That's why thdmerge exists and is used extensively to support Bluray authoring. In case you didn't know, DG invented thdmerge.

Oh, nice, this is what MakeMKV does. That's another thing DG discovered.

There is still an issue with ffmpeg warning that lossless check failed, right? Yes, it's another thing DG addressed. It doesn't make the solution unusable however, and applies to all the tools. It's a warning. Nobody has yet fully reverse engineered THD, so we do what we can.

[Balling]

Except MakeMKV does not have problem on Knives out movie.

"I've investigated the Knives Out issue, and it seems that mlp indeed produces bad output for that disk (and presumably for all "please file issue" disks). The TrueHD streams are actually about 1000 samples shorter than the video segments, according to ffmpeg. But, MakeMKV somehow magically finds the missing audio (and it is legit, it's not just copied to fill the length or something) and gets a correct demux. So for now, until I get to the bottom of this, I'd suggest using MakeMKV for disks where it says "please file issue" at the end."

https://github.com/domyd/mlp/issues/3#issuecomment-739534770

This is obfuscation on the Blu-ray disk itself. And it is hard to get it right. It is a bug in ffmpeg arguably which is used by mlp tool. M2TS parsers had other bugs like when timecode had 0x47 byte when that byte is used for sync, and no one considered M2TS specific header can contain 0x47 too, fixed in https://github.com/FFmpeg/FFmpeg/commit/550c662a1f94ddcd8d91f9874edc4b68d73fa7c7

[Columbo]

Yes, Knives Out has obfuscation, but DGDemux handles it fine when you choose the correct MPLS, which is easy using the GUI. stevew wrote:

Yes, the audio appears to be perfectly in sync and hence it looks like the frames that DGDemux produces are correct. I'm just trying to better understand where these frames actually come from (or why they appear to be missing when counting the frames in the segments).

DGDemux appears to be working fine so please don't waste your money, but if you really want to investigate further this is the disk I have: https://www.amazon.com/Knives-Out-Blu-r ... 081KPWLPR/

no one considered M2TS specific header can contain 0x47 too That was handled correctly in DG tools for almost 20 years.

I'm curious about what you are trying to accomplish here?