DGAVCDec 1.0.9 - Page 89

Guest · 28th February 2009, 04:32

Quote:

Originally Posted by Inventive Software

I ask because it's the standard container for it. After all, MKV was added and that's a non-standard container, despite its popularity.

MKV has a standard. I don't see a lot of MP4 action, so that's why I asked. However, now with Microsoft pushing the segmented MP4 for streaming, it may become more pressing.

I can make my own samples.

aaar9800 · 8th March 2009, 04:37

Hi neuron2,

I tried to merge 37 AVCHD .m2ts files from my canon hf100 camcorder to then index them together with DGAVCIndex.

When I merge them with either tsMuxeR or tsdoctor or pretty much anything else, loading the .dga through avisynth in virtualdubmod shows that it is 55063 frames, with duration reported as 30:37.26. The demuxed audio is 30:37.896

However, when I merge the files with PIXELA ImageMixer 3 (bundled software with the camcorder), the resulting .m2ts file is indexed to report 55077 frames (30:37.73). The audio is 30:37.824, which is good considering the -66ms delay (I guess this comes from the dropped first 2 frames).

When looking through the generated .dgas, on some of the transitions between two different files, the last gop is sometimes broken off in the PIXELA file, with the last frame of that gop reporting as an IDR. In tsMuxeR's file, the last gop is never broken off and it continuous to be a part of the corresponding "sub"-file. In this case the last frame of the gop(which happens to be a key frame) is not part of the index, and av grows out of sync.

I would have no trouble using ImageMixer 3, if it didn't always crash, wasn't so bloated and slow, and didn't require me to unregister Haali Media Splitter every time I needed to use it.

I was wondering if this inconsistency is a result of the way the other programs merge the files, or if it is fixable within DGAVCIndex. If you want, I could send you both .dgas for the different files, or I could try to reproduce the issue with smaller files and send those.

Thanks for all of your great tools.

Guest · 8th March 2009, 05:47

Merge them using DOS: COPY /B.

aaar9800 · 8th March 2009, 17:26

Quote:

Originally Posted by neuron2

Merge them using DOS: COPY /B.

That produced an identical file to what tsdoctor made and the 14 frames are still missing.

Guest · 8th March 2009, 17:31

I'm not sure what you are asking from me.

Are you saying there is some AV sync issue when using the DOS-joined stream?

aaar9800 · 8th March 2009, 17:36

Yes, the audio of the DOS-joined stream ends up 467 ms longer.

moviefan · 8th March 2009, 17:37

I have not dealt with such a situation, but doesn't DGAVCIndex support the functionality to add multiple files whose order can be adjusted if necessary? Thus you could add your m2ts-files, reorder them to the correct order and create the dga-file? Or is there a reason not to do so? (probably a question particularly for neuron2)

Guest · 8th March 2009, 20:02

Quote:

Originally Posted by aaar9800

Yes, the audio of the DOS-joined stream ends up 467 ms longer.

I didn't ask about the lengths. I asked if there is an AV sync problem. There are quite valid reasons for a length discrepancy. It's only an issue if there is an AV sync problem.

If there is an AV sync problem, is it a fixed offset or does it grow larger as the stream is played?

@moviefan

Multiple file open is not currently supported. In any case, it would be equivalent to the DOS COPY operation.

aaar9800 · 8th March 2009, 20:48

Yes, there is an AV sync problem. Starts off fine and then tracks get progressively out of sync.

Turtleggjp · 9th March 2009, 17:26

Yes, this is definitely a problem. I experience the same thing with my video camera footage (also HF100). The reason behind this is simple: Video frames go by at 29.97 fps, while audio frames (Dolby Digital in this case) go by at 31.25 fps. It is pretty much impossible for the audio and video to have the exact same duration for all your individual clips, so what happens is that the audio ends up being slightly longer than the video (by a few ms per clip). After joining several clips together, the audio gets progressively more out of sync.

The solution to this problem is simple, and perhaps Neuron2 could add this ability (as an option) into the program. All that needs to happen is to count the number of audio and video frames that go by with each file. The total duration of both audio and video can be calculated from this, and when the audio difference exceeds 16ms (half a frame), the program can simply drop one frame of audio from the output stream, thus now putting it slightly behind. This is in effect what I do with all my footage.

I use DGAVCIndex to index all my files individually (using the command line interface for batch processing), and then use the program AC3 Cutter to examine all the resulting audio files, counting their frames. This series of numbers is then fed into an Excel spreadsheet that I made up, which then determines where audio frames need to be dropped. I then use tsMuxeR to create a single (out of sync) .AC3 file from all my clips, then use the data from my Excel spreadsheet to cut that file using AC3 cutter. The result is a synced (+/- 16ms) .AC3 file that has not been re-encoded.

If this sounds complicated, it kind of is, at least my way of doing it is. The point is, this proceedure does work, and it would be nice if either tsMuxeR or DGAVCIndex could do this instead. Does this sound feasible to add to DGAVCIndex?

Matt

Guest · 9th March 2009, 17:45

The problem is not with the audio rate, it's with the fact that the video granularity is one video frame=33.367 ms while the audio granularity is one audio frame=32 ms. So the clips will not have the same audio and video length if only complete frames are included. The difference then accumulates and the total error will depend on the number of clips.

Can't you ask your camera to capture in one file? I don't really see this as a job for a decoder.

SeeMoreDigital · 9th March 2009, 19:27

Hi Donald,

I've been meaning to ask you for a while.....

Is there any particular reason why the "Frame Type" displays "Not Yet"?

EDIT: Bummer... I've just found a post (for the release of v1.0.1), where you mention the "Frame Type" function has been disabled

rebkell · 9th March 2009, 19:35

Quote:

Originally Posted by neuron2

The problem is not with the audio rate, it's with the fact that the video granularity is one video frame=33.367 ms while the audio granularity is one audio frame=32 ms. So the clips will not have the same audio and video length if only complete frames are included. The difference then accumulates and the total error will depend on the number of clips.

Can't you ask your camera to capture in one file? I don't really see this as a job for a decoder.

I know this is off topic, but TS Packet Editor is pretty good about keeping timestamps aligned, you can load all the clips in to it and create one ts file, then you could run eac3to to fix the gaps and give you a synced ac3 and raw video file, which you can then mux back together and index or just index the raw 264, depending on your needs.

Turtleggjp · 9th March 2009, 19:57

Quote:

Originally Posted by neuron2

The problem is not with the audio rate, it's with the fact that the video granularity is one video frame=33.367 ms while the audio granularity is one audio frame=32 ms. So the clips will not have the same audio and video length if only complete frames are included. The difference then accumulates and the total error will depend on the number of clips.

That's pretty much what I said, except that you were talking about seconds (ms actually) per frame, and I was talking about frames per second.

Quote:

Originally Posted by neuron2

Can't you ask your camera to capture in one file?

The camera creates a new file every time you start and stop recording. If you record for long enough to make a 2GB file (about 17 minutes if I remember), it will then create a new file due to an apparent 2GB file size limit. When it does this though, these files can be joined by a simple DOS COPY /B without causing sync loss.

Quote:

Originally Posted by neuron2

I don't really see this as a job for a decoder.

A decoder, no. But as something that is demuxing the audio track, it would be nice if it could. I know DGAVCIndex is already counting video frames, since that is what I use it for. Would it be very difficult for it to count the audio frames too? Even if you could have this option for dolby digital tracks only (since that's what most AVCHD camcorders record) it would be very helpful.

Guest · 11th March 2009, 01:58

1. Fix a bug in backward GOP stepping.

2. Fixed a problem with Load Project.

3. Added option "Display HD Full Sized".

4. Fixed a problem in M2TS file parsing.

http://neuron2.net/dgavcdec/dgavcdec.html

Sagekilla · 12th March 2009, 04:17

neuron2, is MKV support available at the moment?

I'd like to try indexing a Blu-ray movie I ripped using MakeMKV (It's the original m2ts muxed into mkv). I can't copy the movie through other methods (Like DumpHD + DumpVID + aacskeys) because they don't work for me, so I used this.

poisondeathray · 12th March 2009, 05:27

Quote:

Originally Posted by Sagekilla

is MKV support available at the moment?

Current Limitations
1. Only AVC/H.264 elementary (raw) and transport streams can be opened (no MKV or MP4 files yet).

Guest · 12th March 2009, 06:17

Quote:

Originally Posted by Sagekilla

I'd like to try indexing a Blu-ray movie I ripped using MakeMKV

Demux the streams from the MKV and then process the elementary video stream in DGAVCDec.

Sagekilla · 12th March 2009, 06:38

I was hoping I wouldn't have to do that since demuxing a 24 GB file is going to take a while. I suppose it's only thing I can do though.

Any idea if mkv will be supported in the near future?

JK1974 · 12th March 2009, 12:44

Quote:

Originally Posted by Turtleggjp

The solution to this problem is simple, and perhaps Neuron2 could add this ability (as an option) into the program. All that needs to happen is to count the number of audio and video frames that go by with each file. The total duration of both audio and video can be calculated from this, and when the audio difference exceeds 16ms (half a frame), the program can simply drop one frame of audio from the output stream, thus now putting it slightly behind. This is in effect what I do with all my footage.

I use DGAVCIndex to index all my files individually (using the command line interface for batch processing), and then use the program AC3 Cutter to examine all the resulting audio files, counting their frames. This series of numbers is then fed into an Excel spreadsheet that I made up, which then determines where audio frames need to be dropped. I then use tsMuxeR to create a single (out of sync) .AC3 file from all my clips, then use the data from my Excel spreadsheet to cut that file using AC3 cutter. The result is a synced (+/- 16ms) .AC3 file that has not been re-encoded.

If this sounds complicated, it kind of is, at least my way of doing it is. The point is, this proceedure does work, and it would be nice if either tsMuxeR or DGAVCIndex could do this instead. Does this sound feasible to add to DGAVCIndex

It seems to be more a problem of TS concatenation that makes problems here.
Have you already tried xport on the concatenated file for demuxing? Similar to ProjectX, this seems to take care of the individual timestamps, so in theory, everything should be kept in sync.

I would also be interested in a solution as I am also a HF100 user - only in PAL land. Haven´t tried before if it happens here also...

9th March 2009, 19:27	#1772 \| Link
SeeMoreDigital Life's clearer in 4K UHD Join Date: Jun 2003 Location: Notts, UK Posts: 12,227	Hi Donald, I've been meaning to ask you for a while..... Is there any particular reason why the "Frame Type" displays "Not Yet"? EDIT: Bummer... I've just found a post (for the release of v1.0.1), where you mention the "Frame Type" function has been disabled __________________ \| I've been testing hardware media playback devices and software A/V encoders and decoders since 2001 \| My Network Layout & A/V Gear \| Last edited by SeeMoreDigital; 9th March 2009 at 19:40.

11th March 2009, 01:58	#1775 \| Link
Guest Guest Join Date: Jan 2002 Posts: 21,901	Version 1.0.9 1. Fix a bug in backward GOP stepping. 2. Fixed a problem with Load Project. 3. Added option "Display HD Full Sized". 4. Fixed a problem in M2TS file parsing. http://neuron2.net/dgavcdec/dgavcdec.html

12th March 2009, 04:17	#1776 \| Link
Sagekilla x264aholic Join Date: Jul 2007 Location: New York Posts: 1,752	neuron2, is MKV support available at the moment? I'd like to try indexing a Blu-ray movie I ripped using MakeMKV (It's the original m2ts muxed into mkv). I can't copy the movie through other methods (Like DumpHD + DumpVID + aacskeys) because they don't work for me, so I used this. __________________ You can't call your encoding speed slow until you start measuring in seconds per frame.

12th March 2009, 06:38	#1779 \| Link
Sagekilla x264aholic Join Date: Jul 2007 Location: New York Posts: 1,752	I was hoping I wouldn't have to do that since demuxing a 24 GB file is going to take a while. I suppose it's only thing I can do though. Any idea if mkv will be supported in the near future? __________________ You can't call your encoding speed slow until you start measuring in seconds per frame.

8th March 2009, 04:37	#1762 \| Link
aaar9800 Registered User Join Date: Mar 2003 Location: Tucson, AZ Posts: 62	Hi neuron2, I tried to merge 37 AVCHD .m2ts files from my canon hf100 camcorder to then index them together with DGAVCIndex. When I merge them with either tsMuxeR or tsdoctor or pretty much anything else, loading the .dga through avisynth in virtualdubmod shows that it is 55063 frames, with duration reported as 30:37.26. The demuxed audio is 30:37.896 However, when I merge the files with PIXELA ImageMixer 3 (bundled software with the camcorder), the resulting .m2ts file is indexed to report 55077 frames (30:37.73). The audio is 30:37.824, which is good considering the -66ms delay (I guess this comes from the dropped first 2 frames). When looking through the generated .dgas, on some of the transitions between two different files, the last gop is sometimes broken off in the PIXELA file, with the last frame of that gop reporting as an IDR. In tsMuxeR's file, the last gop is never broken off and it continuous to be a part of the corresponding "sub"-file. In this case the last frame of the gop(which happens to be a key frame) is not part of the index, and av grows out of sync. I would have no trouble using ImageMixer 3, if it didn't always crash, wasn't so bloated and slow, and didn't require me to unregister Haali Media Splitter every time I needed to use it. I was wondering if this inconsistency is a result of the way the other programs merge the files, or if it is fixable within DGAVCIndex. If you want, I could send you both .dgas for the different files, or I could try to reproduce the issue with smaller files and send those. Thanks for all of your great tools.

8th March 2009, 05:47	#1763 \| Link
Guest Guest Join Date: Jan 2002 Posts: 21,901	Merge them using DOS: COPY /B.

8th March 2009, 17:31	#1765 \| Link
Guest Guest Join Date: Jan 2002 Posts: 21,901	I'm not sure what you are asking from me. Are you saying there is some AV sync issue when using the DOS-joined stream?

8th March 2009, 17:36	#1766 \| Link
aaar9800 Registered User Join Date: Mar 2003 Location: Tucson, AZ Posts: 62	Yes, the audio of the DOS-joined stream ends up 467 ms longer.

8th March 2009, 17:37	#1767 \| Link
moviefan Registered User Join Date: Jul 2005 Posts: 438	I have not dealt with such a situation, but doesn't DGAVCIndex support the functionality to add multiple files whose order can be adjusted if necessary? Thus you could add your m2ts-files, reorder them to the correct order and create the dga-file? Or is there a reason not to do so? (probably a question particularly for neuron2)

8th March 2009, 20:48	#1769 \| Link
aaar9800 Registered User Join Date: Mar 2003 Location: Tucson, AZ Posts: 62	Yes, there is an AV sync problem. Starts off fine and then tracks get progressively out of sync.

9th March 2009, 17:26	#1770 \| Link
Turtleggjp Registered User Join Date: Apr 2006 Posts: 225	Yes, this is definitely a problem. I experience the same thing with my video camera footage (also HF100). The reason behind this is simple: Video frames go by at 29.97 fps, while audio frames (Dolby Digital in this case) go by at 31.25 fps. It is pretty much impossible for the audio and video to have the exact same duration for all your individual clips, so what happens is that the audio ends up being slightly longer than the video (by a few ms per clip). After joining several clips together, the audio gets progressively more out of sync. The solution to this problem is simple, and perhaps Neuron2 could add this ability (as an option) into the program. All that needs to happen is to count the number of audio and video frames that go by with each file. The total duration of both audio and video can be calculated from this, and when the audio difference exceeds 16ms (half a frame), the program can simply drop one frame of audio from the output stream, thus now putting it slightly behind. This is in effect what I do with all my footage. I use DGAVCIndex to index all my files individually (using the command line interface for batch processing), and then use the program AC3 Cutter to examine all the resulting audio files, counting their frames. This series of numbers is then fed into an Excel spreadsheet that I made up, which then determines where audio frames need to be dropped. I then use tsMuxeR to create a single (out of sync) .AC3 file from all my clips, then use the data from my Excel spreadsheet to cut that file using AC3 cutter. The result is a synced (+/- 16ms) .AC3 file that has not been re-encoded. If this sounds complicated, it kind of is, at least my way of doing it is. The point is, this proceedure does work, and it would be nice if either tsMuxeR or DGAVCIndex could do this instead. Does this sound feasible to add to DGAVCIndex? Matt

9th March 2009, 17:45	#1771 \| Link
Guest Guest Join Date: Jan 2002 Posts: 21,901	The problem is not with the audio rate, it's with the fact that the video granularity is one video frame=33.367 ms while the audio granularity is one audio frame=32 ms. So the clips will not have the same audio and video length if only complete frames are included. The difference then accumulates and the total error will depend on the number of clips. Can't you ask your camera to capture in one file? I don't really see this as a job for a decoder.