Joints 1.0.2: joining files with unequal audio/video

[Turtleggjp]

I'm (neuron2) adding a link to Joints here to make it easy to find for newcomers to the thread:

http://neuron2.net/misc/joints102.zip

-----

It could only be to add frames at the end of your files. With the workaround, yes, they get added at different spots but the desync can never be more than one frame at any given time, which is not perceptible.

And to be honest, developing a proper solution for this is very low on my priority list.

I think you're looking at solving this problem the wrong way. The way I do it with my AVCHD camcorder is by removing audio frames instead of adding (repeating) video frames. I did my usual calculations that I do when I have a series of clips to the 3 sample clips he posted. Here are my results:

Clip1:
Video Frames: 3780 (126.126 sec)
Audio Frames: 3942 (126.144 sec)
Excess Audio: ~18ms <-Greater than 16ms (half of one frame) so DROP one audio frame!
Audio is now ~14ms too short.

Clip2:
Video Frames: 1290 (43.043 sec)
Audio Frames: 1346 (43.072 sec)
Excess Audio: ~28ms <-WAIT! Audio is already 14ms behind.
Adjust for previous clip: ~14ms <-Less than 16ms (half of one frame) so Leave this one alone!
Audio is now ~14ms too long.

Clip3:
Video Frames: 810 (27.027 sec)
Audio Frames: 845 (27.040 sec)
Excess Audio: ~13ms <-WAIT! Audio is already 14ms ahead.
Adjust for previous clip: ~27ms <-Greater than 16ms (half of one frame) so DROP one audio frame!
Audio is now ~5ms too short

Presto! Your clips are never more than 16ms out of sync either way. I've been doing this for a little over a year now with my camcorder's footage, and it works great. In fact, I just finished a trip that had about 271 clips.

I use a spreadsheet in Excel to help me set everything up, including using DGAVCIndex to index all my clips and produce an audio file for each one. Using the .dga file for each clip, I can determine the number of video frames. I then run AC3 Cutter on all the audio clips, and it reports the number of audio frames. Once I have this information, all the information I posted above can be calculated. I then have tsMuxeR join all my clips together (with unsyncronized audio) and then have AC3 Cutter go in and trim out all the audio frames that I determined need to be cut out. I can then run tsMuxeR again, but this time have it not use the audio in my clips, but instead use the audio file created by AC3 Cutter. I now have a .M2TS file with all my videos joined together, and with syncronized audio.

If this sounds like a complicated process, well yes it is. I consider it more of a proof-of-concept. Eventually, I'd like to either make up a little tool that would be able to generate the synced audio track, given a series of clips, or else hope that someone like Neuron2 would be willing to add code to his tools to create the synced audio track. All you need to know to do the calculations is:

1. Number of video frames per clip (Indexer already knows this)
2. Video frame rate (Indexer already knows this)
3. Number of audio frames per clip (this could be trickier)
4. Audio frame rate (31.25 fps for AC3 audio, the most common in AVCHD camcorders).

By creating a "camcorder mode" in the indexer, AC3 audio can be assumed for now until more codecs start showing up. That might help figuring out #3.

What do you think Neuron2? I'm willing to help you on this if you need any more info.

Matt

[neuron2]

Yes, I understand that cutting audio is an option, but I assumed it may create objectionable pops.

But anyway, sure, it's possible. First, I'd prefer to do this with a preprocessing tool rather than in the DGNV tools. But second, as I said, it's very low on the priority list for me, especially since I have an acceptable (to me) workaround that doesn't involve the massive effort you are going through.

[Turtleggjp]

Yes, I understand that cutting audio is an option, but I assumed it may create objectionable pops.

I think I understand what you mean by pops. I want to clarify that my method is only useful for camcorder files, not seamless branching Blu Rays like Ratatouille. Files on a seamless branching Blu Ray should naturally combine together (perhaps with a video frame repeat or two) and the audio should not be cut. With camcorders though, the clips do not naturally combine together (since they are usually created from two different scenes) and so there is already a "pop" at the junction.

One other thing, I can think of a couple different types of "pops." One is caused by having a complete audio frame missing, which simply results in missing data. This can be tricky to hear if the audio is quiet. Even if it is loud, it may take careful listening to hear. These "pops" are not too bad, but should probably still be avoided. The other kind of "pop" occurs when a program blindly trims audio data as if it were LPCM. This can result in a very loud pop or chirp, since the compressed data has now been altered significantly. These types of "pops" should definitely be avoided.

This is why I suggested the "camcorder mode" option. My method works for camcorder files, like the ones posted, but should not be used for seamless branching Blu Rays.

But anyway, sure, it's possible. First, I'd prefer to do this with a preprocessing tool rather than in the DGNV tools. But second, as I said, it's very low on the priority list for me, especially since I have an acceptable (to me) workaround that doesn't involve the massive effort you are going through.

Ok, I'll see what I can do about a preprocessing tool. I've wanted to do that for a while now, as I think it would be very useful for people with AVCHD camcorders. You'll have to be patient though, I'm not a highly experienced developer like Neuron2...

EDIT: BTW, it was only a massive effort creating the Excel file that I use as a template. Now, it's just a moderate effort. :)

[neuron2]

How do you automate the actual audio frame cuts? Does AC3 Cutter take a cut list of some sort? What is the link to this cutter tool?

I'm starting to get a little interested. :)

[rack04]

How do you automate the actual audio frame cuts? Does AC3 Cutter take a cut list of some sort? What is the link to this cutter tool?

I'm starting to get a little interested. :)

Speaking of cut list, is there a way to translate the trim/cut points (pressing [ and then ]) specified in the dga/v/m files to a cut list to trim the audio at the same points?

For example:

RANGE 1110441 4582562

[neuron2]

First things first. We're thinking along the same lines, but I want to know about this cutter thing first.

[Turtleggjp]

Found it:

http://forum.doom9.org/showthread.php?t=92325

I actually use a modified version that fixes a couple issues with his parsing of the cut list. I've been using it for a couple years now to cut the audio on my captured TV shows.

Basically, it takes a VirtualDub style .vcf file as its cut list, which my Excel spreadsheet generates for me. For example, here is the one it generated for those 3 sample clips:

VirtualDub.subset.AddRange(0,3941);
VirtualDub.subset.AddRange(3942,1346);
VirtualDub.subset.AddRange(5288,844);

[neuron2]

Reading that thread, the author says it's buggy and shouldn't be used!

Who made your modifed version of it? Is it also buggy? Can we have the source code?

What is the syntax of the AddRange() function?

I'm sure people will ask: What about AAC, LPCM, and MPEG audio? :(

[Turtleggjp]

Reading that thread, the author says it's buggy and shouldn't be used!

I think he was refering to the way he determined which audio frames to take, which was buggy. I figured out how to do it right, and modified the code, which leads to...

Who made your modifed version of it? Is it also buggy? Can we have the source code?

I modified it, as far as I know it is not buggy, and yes here is the source code.

What is the syntax of the AddRange() function?

AddRange(StartFrame, NumberOfFrames), just like VirtualDub.

I'm sure people will ask: What about AAC, LPCM, and MPEG audio? :(

Good question. Do you know of any camcorders that use these? As I said, my method is so far only for AVCHD camcorder clips, not Blu Rays.

Bottom line, AC3 cutter does have a lot of extra stuff in it that I never need, like patching bitrates to a higher one in order to maintain constant bitrate. I basically threw this method together with what I had and filled in the gaps where needed. I figured your tools already know how to pull the audio files out of .M2TS files, so it's just a matter of looking at the audio and counting how many frames are in it. With a little analysis, you can determine when to throw out a frame when joining files together.

Really bottom line, this works great for me, and I would be happy to share what I know to get it working for others as well.

EDIT: Most of my modifications to the code are in the "parseVCF" function in the "ac3functions.c" file. I seem to have commented it pretty well, so I hope you can follow what I did.

[neuron2]

Thanks, Matt. I think I may make something out of this along the lines of the preprocessing utility we discussed above. Automatic cut list generation is a must.

[Turtleggjp]

Absolutely. That's basically what my Excel spreadsheet does is generate that cut list. Prior to that though, it also creates a batch file that calls DGAVCIndex for each clip, and then another batch file that runs AC3 Cutter on all the audio clips created by the first batch file (to have it report number of audio frames for each one). Once the data from these two runs is gathered into my spreadsheet, it can make the cutlist. After that, AC3 Cutter does the actual cutting after tsMuxeR has (blindly) joined all the clips together and then demuxed the audio track.

[neuron2]

OK, first phase completed. I have a working GUI that takes a file list and determines the info needed for the cuts. Tested it successfully on the camcorder files and on Ratatouille M2TS files. The app is tentatively named "Joints".

Next step: integrate the cutter code so that the GUI app can directly correct the already demuxed audio file. Of course that will make the whole thing open source.

[neuron2]

Matt, I've run into a problem with your code. The syntax you give above for your AddRange calls passes the audio frame numbers, e.g. 3942 as the start of the second range. But it appears that these are interpreted as video frame numbers in the code:

FrameTime = 1 / vfps;

Time = ((sFrame + nFrames) * FrameTime) - dDelay + Sync;
RoundUp = ((Time / AC3Time) - (long)(Time / AC3Time)) >= .5;
tPtr->endFrame = ((long)(Time / AC3Time)) + RoundUp; You can see that Time is calculated using the video frame rate! Also, doesn't VirtualDub expect video frame numbering in those calls?

So, please explain. I do not see any reason for the cutter to know about the video frame rate if it is receiving cut data referred to audio frame numbering.

I can fix it to use the audio frame numbering as it should but thought I better mention it so you are aware that you may have been doing things wrongly.

[Turtleggjp]

Matt, I've run into a problem with your code. The syntax you give above for your AddRange calls passes the audio frame numbers, e.g. 3942 as the start of the second range. But it appears that these are interpreted as video frame numbers in the code:

FrameTime = 1 / vfps;

Time = ((sFrame + nFrames) * FrameTime) - dDelay + Sync;
RoundUp = ((Time / AC3Time) - (long)(Time / AC3Time)) >= .5;
tPtr->endFrame = ((long)(Time / AC3Time)) + RoundUp; You can see that Time is calculated using the video frame rate! Also, doesn't VirtualDub expect video frame numbering in those calls?

So, please explain. I do not see any reason for the cutter to know about the video frame rate if it is receiving cut data referred to audio frame numbering.

I can fix it to use the audio frame numbering as it should but thought I better mention it so you are aware that you may have been doing things wrongly.

I thought of this last night. What's going on is that this program was originally designed to work with video frames as cut points, not audio frames. This is because the .vcf files were supposed to come from VirtualDub, which is talking about video frames with its AddRange function. In fact, the code I fixed deals with this conversion from video frame rate to the audio frame rate. This fix is almost unnecessary for the application we are talking about, since when I call this program, I specify a frame rate of 31.25 (AC3 frame rate). This is supposed to make the conversion from video frame rate to audio frame rate a simple 1:1 conversion. I say almost unnecessary, because the program does not recognize the simplicity of the conversion and did all its calculations anyway. Due to the bugs in the code, it would occasionally miss cut the audio by a frame here and there. I built a checking function into my Excel spreadsheet to catch these errors, so I could correct them (This got to be tedious sometimes). Once I corrected the code, these errors have never happend again, but my code still does not do a simple 1:1 mapping of the cut points when a frame rate of 31.25 is specified.

I made my fixes to the code in order to cut audio on HD captures long before I had my AVCHD camcorder. When I finally got my camcorder and realized what was happening when I tried to merge my clips together, I realized I needed a way to cut frames out. This led me back to AC3 Cutter. So as you can see, it's a classic case of using a program for something that it was not exactly intended to be used for.

Anyway, to answer your question, you can probably ignore the code that seems to be converting video frames to audio frames. Whenever I run this program for this purpose, the two are the same.

Matt

EDIT: Here is a sample of one of my command lines:

"E:\Apps\AC3 Cutter v0.4\AC3 Cutter.exe" -fr 31.25 -vcf "E:\Video Camera\AVCHD\090828 - Lake Basin\090828 \
- Lake Basin.vcf" -o "E:\Video Camera\AVCHD\090828 - Lake Basin\090828 - Lake Basin.txt" \
"Q:\090828 - Lake Basin.track_4352.ac3" "E:\Video Camera\AVCHD\090828 - Lake Basin\090828 - Lake Basin.ac3"

[neuron2]

Whoa, you lie about the frame rate!

That's real hacky. I'll have to clean it all up. I assume you are not a professional software developer. :)

[Turtleggjp]

Whoa, you lie about the frame rate!

That's real hacky. I'll have to clean it all up. I assume you are not a professional software developer. :)

Yes it is. You can probably just throw out all that code that converts video frames to audio frames, since you are already calculating the audio frames that need to be taken.

Like I said, this program was not designed for this particular application. Turned out, I was able to get it to work pretty well, and then it turned out that my fixes made it work perfectly. I've been meaning to write a tool like you are doing, but I feel fortunate that I was able to port my "code" over from Excel into a C++ program. As you have guessed, no I am not a professional software developer (yet...).

[neuron2]

OK, I have fixed up the cutter. It is still separate and my GUI writes a cut list.

I applied it to the Ratatouille BD and the result was perfect sync. The process removed 512ms (16 frames) of audio as needed.

I will integrate the cutter into the GUI and then make a beta release.

Matt, I want to commend you and thank you for bringing this idea and process to the fore.

[neuron2]

It's all done and integrated into one convenient GUI.

You first demux the AC3 audio as usual. Then you open Joints, specify the source transport files, select the AC3 file, set your PIDs and frame rate, and then hit Process. When it finishes you have a trimmed AC3 file. A cut list is also written.

I just have to write the help file and then you can have it.

[neuron2]

Here is the first version. Source code will be released after some initial feedback.

http://neuron2.net/misc/joints100.zip

[neuron2]

Now with VC-1 video support:

http://neuron2.net/misc/joints101.zip

[Inspector.Gadget]

I read the documentation included in the 1.0.0 download: is that audio frame length issue present any time a Blu-ray playlist spans more than one M2TS file? Does the Blu-ray playlist structure somehow compensate for this?

Thanks for the tool BTW, it amazes me how quickly the forums' crack tream of developers always comes up with something!

[neuron2]

I read the documentation included in the 1.0.0 download: is that audio frame length issue present any time a Blu-ray playlist spans more than one M2TS file? Yes. Here are the possible video frame times @ different frame rates:

23.976: 41.7 ms
24: 41.67
25: 40
29.97: 33.37
30: 33.33

An AC3 frame is 32 ms. Therefore, because the streams have integral numbers of frames, the probability is very high that the video and audio streams are not exactly the same length.

This is not an issue for BluRay players because they maintain sync using the PTS timestamps. It becomes an issue when you naively concatenate the streams for transcoding or remuxing.

The more files you have the greater the potential problem. For example, on the Ratatouille BD, there are 31 M2TS files in the main movie playlist, and the cumulative desync is 512ms towards the end of the movie. You notice that!

Does the Blu-ray playlist structure somehow compensate for this? No. At playback time the sync is maintained using the PTS timestamps.

[Turtleggjp]

Sorry I haven't been around much today to comment, I'm in the process of experimenting with Linux on my home system, and haven't had an easy way to get to this site all day.

Matt, I want to commend you and thank you for bringing this idea and process to the fore.

Thank you. That means a lot to me coming from a legend like yourself.

Yes. Here are the possible video frame times @ different frame rates:

23.976: 41.7 ms
24: 41.67
25: 40
29.97: 33.37
30: 33.33

An AC3 frame is 32 ms. Therefore, because the streams have integral numbers of frames, the probability is very high that the video and audio streams are not exactly the same length.

This is not an issue for BluRay players because they maintain sync using the PTS timestamps. It becomes an issue when you naively concatenate the streams for transcoding or remuxing.

The more files you have the greater the potential problem. For example, on the Ratatouille BD, there are 31 M2TS files in the main movie playlist, and the cumulative desync is 512ms towards the end of the movie. You notice that!

I'm assuming that for right now this only works with AC3. Since you are also interested in using this program with Blu Ray disks, I'm sure you will eventually run into DTS and other formats that will need similar trimming. LPCM should be easy to trim though.

I'll give this a try tomorrow with one of my sets of footage from my video camera. I already have an audio track trimmed by my method, so I'll see how it compares to what your program creates.

No. At playback time the sync is maintained using the PTS timestamps.

For a brief time when I tried using PowerDVD to play Blu Ray discs, I remember during "Meet the Robinsons" how the video would seem to pause slightly at times during the movie. This is probably how the player maintained sync across multiple files using the timestamps. I don't suppose it would be possible to re-create this type of behavior when indexing a Blu Ray movie with your tools, would it? Perhaps listing a frame in the index file twice would cause it to be decoded twice, thus creating a repeated frame. Like I said before, the technique of removing audio works great for camcorder files where the audio has a natural break, but on a movie like Ratatouille, some people might prefer to not have data missing like that.

[neuron2]

I didn't notice anything in the Ratatouille audio after cutting. But of course the alternative strategy of adding video could be considered. I'll be driven by demand, i.e., based on what crops up in the wild and what users actually ask for based on real cases.

[laserfan]

I guess this is one of those "if you don't know why you'd need it, then you don't need it" tools, but as usual I'm confused about something; in the docs it's stated:

...load the transport stream files into your demuxer and demux the AC3 audio in the
normal way. At this point the AC3 file is not synchronizeded to the video, due to the presence of the
excess audio. Joints will create a new AC3 file with the excess audio deleted.

If my "normal way" of demuxing is to use eac3to, which not only demuxes but also re-joins (to make a single continuous audio track), then is its re-joining defective? neuron2 I know you don't like to comment on others' tools, but if you didn't use eac3to to demux Ratatouille then what do you use normally? :confused:

[neuron2]

I use DGIndex (and NV variants) of course!

I doubt that eac3to adjusts the audio to account for these joins, but I haven't tested it. I'll test it and report back.

[laserfan]

I use DGIndex (and NV variants) of course!...oops... :o

Thanks for indulging me... :)

[Turtleggjp]

I use DGIndex (and NV variants) of course!

I doubt that eac3to adjusts the audio to account for these joins, but I haven't tested it. I'll test it and report back.

Actually, I think it does. I remember using it on The Simpsons Movie (DTS Master Audio) and it did some adjustments on that. The movie was only in 4 pieces though, so it wasn't nearly as complex as Ratatouille. I do remember using eac3to for Ratatouille though, and it did do a good job at cutting the AC3 audio.

[Turtleggjp]

My initial test today was with a series of 132 clips from my camcorder. At first, it did not make any cuts, and the cuts file had only 0,0 in every line. Turned out, my audio PID was 1100 instead of the default 1101. After correcting this, the cut list now matches what I was getting when I created my .vcf file using my old method. However, the result is still exactly the same size as the original (about 2 seconds and 59 KB should be removed). I'm going to try it again with a shorter series of clips right now. Will report back later...

[neuron2]

However, the result is still exactly the same size as the original (about 2 seconds and 59 KB should be removed). Sorry I can't believe that. You're going to have to offer me a set of files that creates this claimed problem.

[Turtleggjp]

Tried another series of clips, this time only 19. Now it is actually cutting, and the results are almost in 100% agreement with what I created with my method. The files are identical in size (the one created by Joints and the other one created by my old method), and your cut list matched mine perfectly. Where we disagree is when I do a binary comparison of the files. There seems to be 7 places where it looks like a single AC3 frame does not match. I'm guessing that this is at the cut points, because checking my Excel file for that batch, there are 7 times when a frame needs to be removed. So the question is, do you drop the last frame of the previous clip, or the first frame of the next clip? Either method will work, but if we are doing things differently, that would explain why the files are not 100% identical.

Oh, and a little feature request: Since most camcorder clips are in sequence, it would be nice if you could give file names in the input list file as a range. For example:

00788.m2ts - 00919.m2ts

That is my range for the 132 clips I first tried. Multiple ranges would be nice too, for example my other big trip would look like this:

00137.m2ts - 00260.m2ts
00375.m2ts - 00521.m2ts

That would certainly make things easier for us camcorder users.

[Turtleggjp]

Sorry I can't believe that. You're going to have to offer me a set of files that creates this claimed problem.

Just ran it again from scratch (freshly opened the Joints program) and now it is cutting. It looks like it was some sort of initialization problem, because when I was having trouble, I had already run the program once with the wrong audio PID, and then I just ran it again with the correct audio PID but without closing and re-opening the program first. So probably what happened was that the old cut list was still being used instead of the newly generated one.

Your file is still 4KB bigger than mine, but I think it might be a problem with my method. I noticed in my number of frames to cut column that there was a -3 value instead of the usual 0 or 1. Turns out, that particular clip was the one that DGAVCIndex Alpha 23 choked on (saw it as a 188-byte packet stream instead of 192), and I had to index it with a later version, which probably dropped some of the audio.

Anyway, it is looking very good so far! When I get a chance, I'll run it on that huge series of clips I mentioned earlier.

EDIT: Just fixed my glitch on the 132 clip series. I'm still 1KB (exactly 1024 bytes) smaller than you. Still investigating...

EDIT2: No big deal, you just have one extra audio frame at the end of your file. Turns out a 256kbps AC3 audio frame is exactly 1024 bytes.

[neuron2]

It looks like it was some sort of initialization problem, because when I was having trouble, I had already run the program once with the wrong audio PID, and then I just ran it again with the correct audio PID but without closing and re-opening the program first. So probably what happened was that the old cut list was still being used instead of the newly generated one. Indeed. I need to reinitialize the cut list when the process button is hit. Thanks for pointing it out.

I have an extra frame at the end? Is it listed as a cut in your spreadsheet output? Can't cut it on a whim. :)

[Turtleggjp]

I have an extra frame at the end? Is it listed as a cut in your spreadsheet output? Can't cut it on a whim. :)

I'm not sure where the extra frame is coming from right now. As far as I can tell, our cut lists match up 100%. I'll look into whether or not my method is somehow dropping the last frame from the last clip. Should be easy to figure out.

[neuron2]

OK, thanks. I'll add your range idea also.

[Turtleggjp]

OK, thanks. I'll add your range idea also.

Excellent.

BTW, I ran my big set of clips (271 clips, 15.6GB of data, 2 hours, 18 minutes, 51 seconds of video) and the results were the same as the last set, that extra frame (1024 bytes) is there at the end of your file. I'll look into this shortly...

EDIT: It looks like my method is dropping the last frame sometimes, while your program is not. I'm not sure why it is dropping the last frame because according to my spreadsheet it should not be. Probably just another bug in the original AC3 Cutter code. Anyway, it looks like the Joints program is correct.

EDIT2: I just looked into the other area where our files are different. When a frame gets dropped, the two files were different for the duration of one frame. I thought it was because I was dropping the last frame of the first file and Joints was dropping the first frame of the second file. Turns out, I was actually dropping the second to last frame of the first file and keeping the last frame. This is where the files did not match. The next frame was the first one of the next clip, and the two files once again match. Must be a problem with the way AC3 Cutter numbers the audio frames. So once again, Joints is correct. This is probably related to the problem of the last frame getting dropped with my method.

[Efenstor]

Yes, it works perfectly! Though I must use a filelist printing software (PrintFolders) to create the list of files, would be better to use multiple selection in GetOpenFileName. :) One thing also missing in the readme: the new line at the bottom of the list is compulsory. Saving the options in an INI also would help (my camcorder uses 1100 for audio). The rest is perfect.

[Turtleggjp]

Yes, it works perfectly! Though I must use a filelist printing software (PrintFolders) to create the list of files, would be better to use multiple selection in GetOpenFileName. :) One thing also missing in the readme: the new line at the bottom of the list is compulsory. Saving the options in an INI also would help (my camcorder uses 1100 for audio). The rest is perfect.

That's great news! I'm glad if it helps you so much. I'm sure all of your suggestions will be added to the program. Right now, he just wanted to get it out there and let those of us with camcorders start messing with it. Did you see my suggestion about input ranges? Neuron2 said he was going to add that, so that should make things a lot easier right there.

[neuron2]

All suggestions will be implemented. Thank you both for your valuable ideas.

[Turtleggjp]

One more request. It would be beneficial to me at least if Joints could also output the number of video frames for each clip to a file. A simple list like this would work:

00137.m2ts 1125
00138.m2ts 645
00139.m2ts 675
...

This would help me because I will still want to use my spreadsheet to create chapter points in my videos, and I'll need the number of video frames for each clip for that. This could be an option in the GUI, since I doubt it would be of much use for Blu Rays. Thanks!

[neuron2]

Can I also print the number of audio frames or will that get in the way for you? I'd just add the info to the cuts.txt file.

Give me a few days. I've just finished up the INI file support. Want to work on my quantum physics simulations for a while. (BTW, my paper demolishing Bell's Inequality was accepted by Physics Essays and will appear in December.)

[neuron2]

Here's how the cuts.txt will be:

00066.mts [3780/3942]: 0,3941
00067.mts [1290/1346]: 3942,1346
00068.mts [810/845]: 5288,844

The numbers in brackets are the vid frame count / aud frame count. This is good because you can see where the frames were deleted by comparing the audio frame count to the audio block length.

Will that serve your needs?

[neuron2]

Just dashed off a new version:

* Adds range support to the playlist file (read help file!)
* Outputs file name and frame counts in the cuts.txt file.
* Adds INI file support.

http://neuron2.net/misc/joints102.zip

I probably won't support any other file open method because the dir /B trick described in the help file is so easy, and because we have ranges now.

[neuron2]

I jst added the source code to the 1.0.2 zip to satisfy GPL.

[neuron2]

If my "normal way" of demuxing is to use eac3to, which not only demuxes but also re-joins (to make a single continuous audio track), then is its re-joining defective? I just tested eac3to on a set of the files at issue. It does NOT correct for this situation and its demuxed audio is binary identical to the uncorrected audio demuxed by DG tools.

If you have this problem you will certainly know about it.

Interestingly, on the web there is a lot of discussion about the sync issue with Ratatouille BD. Looks like we're the first guys to realize what the cause really is and how to fix it. Score another one for Doom9 forums!

[laserfan]

I just tested eac3to on a set of the files at issue. It does NOT correct for this situation and its demuxed audio is binary identical to the uncorrected audio demuxed by DG tools.

If you have this problem you will certainly know about it.

Interestingly, on the web there is a lot of discussion about the sync issue with Ratatouille BD. Looks like we're the first guys to realize what the cause really is and how to fix it. Score another one for Doom9 forums!I don't do a lot of BD backups/conversions and OTTOMH can remember only Enchanted had a lot of m2ts files in the set--seemed to convert OK tho. In any case thanks for following-up wrt eac3to! Can't begin to tell you how impressed I am w/you and Turtleggjp for your pursuit/quickness in devising yet another nifty tool! Congrats and

:thanks:

[Turtleggjp]

Here's how the cuts.txt will be:

00066.mts [3780/3942]: 0,3941
00067.mts [1290/1346]: 3942,1346
00068.mts [810/845]: 5288,844

The numbers in brackets are the vid frame count / aud frame count. This is good because you can see where the frames were deleted by comparing the audio frame count to the audio block length.

Will that serve your needs?

I don't need the audio frames for doing my chapters, but they don't hurt, and it would be nice to have them for completeness. I think the brackets and colon throw me off more. I like to import the file into Excel as a text file, and when I do it gives me options for separating things into columns. I can tell it to use spaces, commas, and the '/' to delimit the columns, but that still leaves me with "[3780" in one column and "3942]:" in the next column. While your method looks nicer, I think what would work best for me with Excel would be just this:

00066.mts 3780,3942 0,3941
00067.mts 1290,1346 3942,1346
00068.mts 810,845 5288,844

Since I've discovered that flaw in my processing of the audio, I'll be re-doing a lot of clips in the near future, so I'll be giving this tool a good workout. Thanks again!

[neuron2]

I can't get excited about changing the log file format to allow for Excel's shortcomings. On the other hand, I don't want to leave you stranded. Decisions, decisions...

[neuron2]

Re-download 1.0.2. I revised the log file format to make you happy.

[Turtleggjp]

Just finished running Joints 1.0.2 on my large set of clips, and with the aid of your new log file format, I was able to confirm that all your numbers matched mine! Very happy with this little tool. :thanks:

Just a few more suggestions, no real rush on these as far as I'm concerned. By all means, work on your real work first before trying to tackle these:

1. I think it would be helpful if your indexers could read in the same file that Joints does for the list of files. This way, we can use the same list for indexing (and demuxing) as well as repairing.

2. Blu Ray Playlist support. This would also be nice to have in the indexers as well. This could require a bit more effort to implement, but there might already be code out there to help.

3. This is more of a note to the users than a feature request. Most camcorders will split a clip into multiple pieces if the file size exceeds 2GB. Why? I don't know. I've seen Blu Ray files larger than 25GB, and the FAT32 file system used on the camcorder cards supports up to 4GB files. Anyway, this situation is less common because you usually have to shoot continuously for over 15 minutes to exceed 2GB on a camcorder. Still, it can come up while filming things like school concerts or plays. If you have clips like this they should NOT need special processing by Joints, and can simply be combined with a DOS copy command like this:

copy /b "00002.mts"+"00003.mts"+"00004.mts"+"00005.mts" "L:\Gala.m2ts"

That was the command I used to combine 4 clips together into one. I'm not 100% sure this will work with all camcorders, but it worked with mine. Therefore, I'd suggest that you combine clips like this prior to processing with Joints, and then use the resulting file with your other smaller clips in Joints if needed. @neuron2, if you are interested in adding support into Joints to check for this condition, I have some sample clips you can look at. They are 2GB in size, so maybe you would only need the first and last few MB or so of the file. Let me know if you are interested in this.

:thanks: again!

[Efenstor]

That's cool!
Donald, do you sleep at night? Looks like you're coding without a break.
Take a break.

[neuron2]

If you have clips like this they should NOT need special processing by Joints I'd like to see a sample please. If the streams are just broken binarily in an arbitrary way like VOBs, then it is an issue. You can give me just the start of such a file that was created after an automatic split.

[Turtleggjp]

I'd like to see a sample please. If the streams are just broken binarily in an arbitrary way like VOBs, then it is an issue. You can give me just the start of such a file that was created after an automatic split.

Ok. When I get home tonight I'll upload a sample. Would you like to see the end of the file before it as well?

[neuron2]

Ok. When I get home tonight I'll upload a sample. Would you like to see the end of the file before it as well? That would be useful, so yes please.

[Turtleggjp]

Uploading the samples now...

I couldn't help taking a look at the data as I was cutting it, and I noticed some things that might help:

1. The first thing I noticed is that at the end of the regular clips are several null packets (PID 0x1FFF). The 2GB file also has these null packets, and the new file also begins with some null packets. I do not see any null packets at the start of any of my other clips.

2. Each new file created when I start recording has the extra 4 bytes at the front of each packet start with 0x00000000. In the case of the sample I'm uploading where the file was created after a 2GB split, the first 4 bytes are 0x31EBAFE5. The first 4 bytes in the last packet of the first file are 0x31EBA715. They are very close, and are most likely sequential.

3. The continuity counter on both the video and audio PIDs always seem to start at 0 with new clips. As far as I can tell, the continuity counter for both PIDs is continuous across the file split.

Just some food for thought while the files finish uploading...

[neuron2]

The four byte field is a counter, so it should increment.

It sounds like a clean cut, so there should be no problem joining it with Joints. Why do you think it is a problem?

[Turtleggjp]

The four byte field is a counter, so it should increment.

It sounds like a clean cut, so there should be no problem joining it with Joints. Why do you think it is a problem?

My point is that it is a clean cut, and no special processing is necessary, nor should even be attempted. I remember reading over on dvinfo.net that people were experiencing a gap in the audio when importing these clips into their video editors (Adobe Premiere, Final Cut Pro, etc.). It was driving them nuts because the only way it played back perfectly was on the camcorder! Finally, some "genius" suggested just copying the two files together with "copy /b" and it was like a miracle to them! All I could think of while reading that thread was "duh!"

Anyway, here they are:

Last ~10MB of first clip: http://www.mediafire.com/?lkmjmzvtnm2

First 10MB of second clip: http://www.mediafire.com/?jnzdxkqxgjf

If you join them with this DOS Copy command:

copy /b "00002 (10MB).MTS"+"00003 (10MB).MTS" "L:\Gala test.m2ts"

the result plays nice and smooth over here.

Looking at the raw data again, the last non-null PID packet in the stream has PID 0x1100 which is the audio track. The packet does not have an adaptation field, which would suggest that the audio data did not come to a natural ending point (or the end of a PES packet). If Joints were to conclude its processing of the file at this point, it would probably not have a complete audio frame. A quick check of a typical file from my camcorder shows that the last packets for both the audio and video PIDs in the stream both have the adaptation field in them. It's my understanding that these adaptation fields in the transport packets are there to pad the packets with extra space after the PES packet has ended.

Anyway, you have the samples now, so you can draw your own conclusions...

[neuron2]

Until you tell me theoretically why there should be a problem, or better yet demonstrate a problem!, when processing with Joints I can't get excited about this. Joints will simply see no correction needed at that joint.

Bear in mind that when loading all the files into DGIndex they are combined in an identical way to the DOS COPY /b way. So the only difference could be if Joints does or does not detect a correction needed at that joint.

[Turtleggjp]

Ok, I'm not 100% sure it will cause a problem, I just wanted to bring it up because I saw it as a potential problem. Those samples are from a series of 4 clips. I will run Joints on all 4 and see what the results are.

EDIT: Here is the log from Joints on these 4 clips:

00002.mts,29535,30796,0,30796
00003.mts,29310,30561,30796,30561
00004.mts,29610,30875,61357,30875
00005.mts,1485,1550,92232,1549

It looks like it worked just fine, so I guess you are correct in that this won't be a problem. I think it would be more of a problem if Joints was actually extracting the audio from the clips, but since it is just counting frames, it should be ok.

[Turtleggjp]

@neuron2. Were you able to use a version of eac3to that was able to cut the AC3 audio in Ratatouille? I just used eac3to version 3.05 on Meet the Robinsons (another seamless branching title) and was comparing the results to what Joints produced. The two programs produce almost the same result for me. Joints' file was one frame shorter than eac3to's, but that was just because Joints cut the very last audio frame, and eac3to didn't. The two files start and end in sync with each other (except for the extra frame at the end of eac3to's file), but at times they are not the same. It looks like one program cuts a frame when the other doesn't, de-syncing them, then at the next joint, the opposite happens, and they are again synced (I'm talking about the files' sync with each other, not lip sync). I'll have to do some closer examining to find out who is doing what, and even trickier to determine will be who is right and who is wrong (if that even matters, we're talking about 32ms of sync here). Anyway, I was just curious to know if the same thing was happening with Ratatouille.

[neuron2]

I just used eac3to version 3.05 on Meet the Robinsons (another seamless branching title) and was comparing the results to what Joints produced. Please give me a link to that version of eac3to and tell me the exact command line you used.

[Turtleggjp]

You can get the latest version of eac3to in this thread:

http://forum.doom9.org/showthread.php?t=125966

If you want the exact version I used, I'm uploading it here:

http://www.mediafire.com/?242wzfuwnym

The command line I used is here:

"E:\Apps\EAC3to\3.05\eac3to.exe" f: 1) 1: "L:\Blu Ray\Meet the Robinsons\Chapters.txt" 2: "L:\Blu Ray\Meet the Robinsons\Robinson.mkv" 3: "L:\Blu Ray\Meet the Robinsons\Robinson.flac" 3: "L:\Blu Ray\Meet the Robinsons\Robinson (from PCM).ac3" -640 4: "L:\Blu Ray\Meet the Robinsons\Robinson.ac3" -keepDialNorm 7: "L:\Blu Ray\Meet the Robinsons\Robinson1.ac3" -keepDialNorm 8: "L:\Blu Ray\Meet the Robinsons\Robinson2.ac3" -keepDialNorm 9: "L:\Blu Ray\Meet the Robinsons\Robinson1.sup" 12: "L:\Blu Ray\Meet the Robinsons\Robinson2.sup"

May need some slight modifying for Ratatouille. I can help you with this if needed.

[neuron2]

Couple questions:

1. Why is your input an MKV file? We are talking about sourcing multiple M2TS files!

2. What is that "f: 1)" thing in the options?

Please just explain your command line so I can understand what you are doing with it. I am not an eac3to user.

[Inspector.Gadget]

F: is the input - presumably a disk below AnyDVDHD or a mounted image.

The MKV file is the video output: in eac3to-land, 2: is the video track (1: is the chapters) and eac3to uses Haali's dsmux to dump the main movie video (or that of another playlist, with appropriate options) to an MKV file.

[neuron2]

What is 1)?

[Inspector.Gadget]

1) I believe designates the playlist to encode. eac3to does parsing similar to BDInfo: it lists all substantial playlists on the disc, starting with the largest (usually main movie). Here, I think the 1) is redundant: both "eac3to F: 1) ..." and "eac3to F: ..." would have worked here, and I always do the latter.

[neuron2]

OK, thank you, sir.

[Turtleggjp]

I think he explained it pretty well, but I'll give my explanation too:

"E:\Apps\EAC3to\3.05\eac3to.exe" f: 1) 1: "L:\Blu Ray\Meet the Robinsons\Chapters.txt" 2: "L:\Blu Ray\Meet the Robinsons\Robinson.mkv"\
3: "L:\Blu Ray\Meet the Robinsons\Robinson.flac" 3: "L:\Blu Ray\Meet the Robinsons\Robinson (from PCM).ac3" -640 \
4: "L:\Blu Ray\Meet the Robinsons\Robinson.ac3" -keepDialNorm 7: "L:\Blu Ray\Meet the Robinsons\Robinson1.ac3" \
-keepDialNorm 8: "L:\Blu Ray\Meet the Robinsons\Robinson2.ac3" -keepDialNorm 9: "L:\Blu Ray\Meet the Robinsons\Robinson1.sup" \
12: "L:\Blu Ray\Meet the Robinsons\Robinson2.sup"

My source is "f:" F is my Virtual Drive, so I am basically pointing to the root of a Blu Ray disc. Running eac3to with this alone give a list of "playlists" or "titles." These are numbered as 1), 2), 3), etc. I chose "1)", as that is the main movie in English. Running eac3to with these two commands gives a list of "items" listed as 1:, 2:, 3:, etc. Each item can be a list of chapters (usually the first one), video tracks, audio tracks, or subtitle tracks. What follows next in the command line is a list of all the items I want eac3to to extract. You asked about the MKV in the command line, that was me telling eac3to to extract the video track and mux it to a .MKV file. All you need to be concerned with for the sake of comparison with Joints are items 4:, 7:, and 8:. These are the three AC3 soundtracks. 4: is the main english soundtrack, 7: is the english commentary track, and 8: is the soundtrack only with no dialogue. When I did my comparison with Joints earlier, I only looked closer at item 4:, but the others look very close to eac3to's output as well, probably just missing the last frame.

[neuron2]

I've done some testing and understand it all now.

First, if you just use "eac3to -demux" with a list of M2TS files as I did before, it does not do any gap adjustment. But if you use the track decoding syntax, e.g.:

4: "rat.ac3"

then the gap adjustment is performed and the result is equivalent to Joints.

So Joints is superfluous unless you do not have all the software packages that are needed to run EAC3TO.

Now I'm thinking that Joints should demux the audio and adjust it on the fly as EAC3TO does, eliminating the need to demux it first with DGIndex.

And then the Joints functionality should really be integrated transparently into DGIndex.

So much to do, so little time...

[laserfan]

I've done some testing and understand it all now... So much to do, so little time...Don't know how you get to it all! :eek:

Thanks for following-up with eac3to and reporting back here. I for one appreciate your (and the OP's) unflagging attention to "get everything right". :cool:

[Turtleggjp]

To me, the most ideal solution is to have programs like eac3to, DGIndex, and even tsMuxeR do what joints is doing. I think I asked for this a long time ago in the tsMuxeR thread (back when development was stalled at 1.8.4). Eac3to seems to handle Blu Rays just fine, it was just lacking support for my camcorder files. Hopefully by having Joints out there with its source code, more programs will support such things.

@neuron2, Would you be able to integrate Joints' code into your NV tools without any GPL hassles? I only ask because for AVCHD camcorders, having Joints' functionality in regular DGIndex won't do much good, as it only works on MPEG2.

[neuron2]

Eac3to seems to handle Blu Rays just fine, it was just lacking support for my camcorder files. It should work fine on your files. Why do you say it doesn't.

eac3to file1.m2ts+file2.m2ts 4: "output.ac3"

Would you be able to integrate Joints' code into your NV tools without any GPL hassles? The only hassle-free way would be to get permission from the license holders for the AC3 code I used. But it would not be too hard to rewrite that part using my own coding. It just fetches frames one-by-one from the file and then either writes them or not. I already have AC3 parsing code of my own that I use for demuxing.

[Turtleggjp]

It should work fine on your files. Why do you say it doesn't.

eac3to file1.m2ts+file2.m2ts 4: "output.ac3"

I thought I did try that a while ago. I will try it again and see if it works

[Prthivi]

Neuron2, is this cut and join work on DVD that the audio not sync when demux and remux because of silence?

[neuron2]

You need to cut the silence at the beginning.

Open the VOB(s) in DGIndex. Hit > to skip the black video. Then hit [ to start the project there. Continue as usual.