Primary knowledge of multimedia required to debug tsmuxer?

[yellowolf]

There are no perfect or completely reliable software to demux Blu-ray disk at present. The features of the disks at hand is various, such as 3D, UHD, seamless branching, and so on. I mainly need the demuxed audio/subtitle stream and focus on it, and have tried eac3to, tsmuxer, mkvtoolnix, and ffmepg, only to finding that they all have some issues. The common issue is the demuxed AV are out of synchronous mainly due to the shorten or lengthen duration of audio streams. Only minor part of the disks can be demuxed without any problem.
Rather than continuously reporting issue and wait somebody to pay attention to for a long time, I would like to do something, such as debug, to contribute to the newly open-source tsmuxer. Any way, it seems the most dedicated demux/remux one, and it is in rapid development by several warm-hearted people.
However, my expertise is not the multimedia and even not the code. I just can read C language with slow speed, and know how to debug simple issue with IDE. I ever have spent 3-4 days to figure out the issue of random code when parsing mpls file and then commit a patch on github after learning how to use git. But when I come in the AV desynchronous problem, I feel totally blank. I want to make the accurate synchronous reach to frame. What preliminary knowledge on multimedia are required to debug the demux issue of tsmuxer? Any help or recommendation on good materials would be deeply appreciated. Bow~

[videoh]

Originally Posted by yellowolf

Rather than continuously reporting issue and wait somebody to pay attention to for a long time

Your several issues with DGDemux were addressed immediately and your outstanding issue is awaiting your provision of materials. It's not fair for you to so casually slag off developers (not just myself) that are actively working to improve their software.

[yellowolf]

Originally Posted by videoh

Your several issues with DGDemux were addressed immediately and your outstanding issue is awaiting your provision of materials. It's not fair for you to so casually slag off developers (not just myself) that are actively working to improve their software.

Sorry for the misunderstanding here. I have reported several bugs here and there, not only at DGDemux forum, and not only related to 'demux'.
In fact, it is indeed unfaire to mention it without any clarification when talking about AV desync. The response to bug report from DGDemux is rather rapid which can always give me surprise (usually overnight), and Rocky is a nice man who will mediate accidental aggressive post at forum. Thanks to his dedication to this great project. That's why I decide to donate immediately, introduce to others. And report bugs more frequently than do this at other forum. It seems that DEDemux has become the most accurete demux tool at present especially when demux some seamless branching materials.
With regarding me, I am volumteer to test all kind of tools, and I would like to contribute a little to them in various forms.

[videoh]

Thank you for the clarification.

The first thing you will need to look into is what are transport streams and how they have timestamps to say when a given video/audio unit is supposed to be displayed. Those are the things that allow for in-sync playing by TVs, set-top boxes, and players. However, when you demux you lose all those timestamps so that is the basic challenge. There are a lot of complications. To really do the job well you have to be persistent to address the last few % of the job. That was a lesson I learned early, projects are made or broken based on that last few %. And Murphy's Law dictates that the last few % will be the most difficult and challenging things you'll have to do.

[tebasuna51]

Originally Posted by yellowolf

...The common issue is the demuxed AV are out of synchronous mainly due to the shorten or lengthen duration of audio streams...

Like videoh say you in a container the problem is solved with the timestamps, but the demuxed streams can lose the sync.

See the attached image in https://forum.doom9.org/showthread.p...94#post1600694

The 0.m2ts can be a credit and have a audio length greater (or shorter) than video length.

In (A) the next 1.m2ts begin with a new audio stream than play at timestamp 400 ms. But if you extract the audio streams this new audio begin at 416 ms, the 16+ overlap can't be corrected, if you delete the frame 13 of first audio you have a gap of -16ms.

To preserve the sync you need decode the streams, cut the 16 ms and recode all.

[videoh]

Very interesting link, tebasuna51! May I ask what your definition of a duplicated frame is? Is it that the audio frames are bit-for-bit identical, or is it that they carry the same PTS timestamp?

The perceptibility of desync is not very sensitive (it also depends on whether the audio is ahead or behind). Some people are more sensitive than others as well. The limits obtained from research are way higher than the limits specified by standard.

https://ieeexplore.ieee.org/document/4599253
https://core.ac.uk/download/pdf/55846182.pdf
https://www.itu.int/dms_pubrec/itu-r...2-S!!PDF-E.pdf

At my former employer we required our sync algorithms for STBs to limit things to maximum 20ms desync. So one could argue that it's not worth worrying too much about 16ms.

[tebasuna51]

My soft to do the test in the related post was only over extracted ac3, then the duplicated frames detected was bit-identical.

But, of course, we can use timestamps to decide when delete extra frames.

The big audio frame is for AC3 code with 32 ms, then we can guarantee +-16 ms of async, for me is unnoticeable because is less than the duration of a video frame (40 ms at 25 fps). With DTS we can obtain +- 6 ms.