DGAVCDec 1.0.9 [Archive] - Page 36

lexor

17th February 2009, 15:26

You can cut a segment with DGSplit and upload that to a hosting site.

But why do you talk about an M2TS and then give me a script with:

AVCSourceInContainer.mkv.GRF

???
He mentioned eac3to, mkv is what it like to put video in. The script is part of his post edit and it contains attempt at a workaround. I don't think the two parts of his post are directly related.

magic144

18th February 2009, 03:18

ok, I didn't realize you could cut clips for upload at arbitrary byte locations, but fair enough!

here is a cut-down version of the problematic source .m2ts clip
http://www.mediafire.com/?dcbn0n2dzjz
the glitch(es) using DGAVCIndex 1.0.8 is/are seen to occur right about the middle of the clip

as to my script, yes lexor has it right - that would be a script I would propose to try IF DGAVCIndex was shown not able to handle this clip properly (for whatever reason, e.g. libavcodec)

I demux all my discs with eac3to, so typically I would have a single .h264 file for the complete title which I would preprocess via DGAVCIndex, then frameserve into x264 via an .avs script, using the MeGUI front-end.

a typical script would look like this:-
LoadPlugin("DGAVCDecode.dll")
AVCSource("00011.dga")
crop( 0, 140, 0, -140)
LanczosResize(1280,528) # Lanczos (Sharp)

so I was wondering what difference, if any, I could expect if I demuxed the title as an .mkv using eac3to and fed that into x264 (via a DShow .grf so I knew and could control the filter chain) via the other script I suggested above - I know DGAVCDecode would supposedly make a difference if I was doing something in the script that involved seeking, but I in a straightforward linear encode, would I expect any difference?
(specifically note, I'm falling back on .mkv here because I don't know how to incorporate a raw/elementary .h264 stream file into a DShow graph)

thanks again for having a look

m

ps - any particular reason why DGAVCIndex couldn't/shouldn't be re-built with the most recent stable source-code-base for the libavcodec version that ships with ffdshow tryouts (Beta 6) rev 2527 Dec 17 2008 (since I've seen that that version renders this clip OK)?

Guest

18th February 2009, 03:49

here is a cut-down version of the problematic source .m2ts clip It plays fine in DGAVCDecNV and CoreAVC. It bombs in VLC. So say thank you to libavcodec.

so I was wondering what difference, if any, I could expect if I demuxed the title as an .mkv using eac3to and fed that into x264 (via a controlled DShow .grf so I knew and could control the filter chain) via the other script I suggested above - I know DGAVCDecode would supposedly make a difference if I was doing something in the script that involved seeking, but I in a straightforward linear encode, would I expect any difference? For linear decoding, DirectShowSource() should be fine.

Another alternative of course is to use DGAVCDecNV.

ps - any particular reason why DGAVCIndex couldn't/shouldn't be re-built with the most recent stable source-code-base for the libavcodec version that ships with ffdshow tryouts (Beta 6) rev 2527 Dec 17 2008 (since I've seen that that version renders this clip OK)? It's been discussed to death. There are regressions that break accurate random access that the developers are uninterested in fixing, as they apparently care only about playback and not frame accurate seeking.

magic144

18th February 2009, 04:02

It plays fine in DGAVCDecNV and CoreAVC. It bombs in VLC. So say thank you to libavcodec.

For linear decoding, DirectShowSource() should be fine.

Another alternative of course is to use DGAVCDecNV.

It's been discussed to death. There are regressions that break accurate random access that the developers are uninterested in fixing, as they apparently care only about playback and not frame accurate seeking.

thanks for the verification DG...

(and v. sorry for not having picked up on the discussions - obviously have my fingers in far too many technical pies at the moment and am feeling somewhat overwhelmed! - but I do appreciate your restating the facts)

in which case I guess it's the end of the road for DGAVCIndex (libavcodec version) unless and until the libavcodec developers share your purpose! - that's too bad

yeah, I would buy DGAVCDecNV right now IF I currently had an NVidia graphics card (sadly I'm running with ATI HD3650) - maybe I'll have to go down that road eventually - either that or the software approach with CoreAVC... did I read you were going to consider developing DGAVCIndex variants with ATI or CoreAVC in mind???

(I notice now that CoreAVC has even released a version now that works in an accelerated mode with NVida GPUs!!!)

otherwise, I guess I'm left with DirectShowSource and either *that* version of libavcodec or CoreAVC

cheers once more for all the help and great tools and food for thought

m

Guest

18th February 2009, 04:07

in which case I guess it's the end of the road for DGAVCIndex (libavcodec version) unless and until the libavcodec developers share your purpose! - that's too bad Not necessarily. I'm working on using CoreAVC as the underlying decoder. I don't know if it will be successful.

magic144

18th February 2009, 04:11

well, if anyone can do it... :)

I will certainly follow any progress with interest

MatMaul

25th February 2009, 11:42

ffmpeg have been fixed for PAFF AVCHD files.
it also now parses the SEI recovery points (http://svn.ffmpeg.org/ffmpeg?view=rev&revision=17109).

Inventive Software

27th February 2009, 23:23

Long shot, but any news on whether MP4 support is coming or not?

Guest

28th February 2009, 00:12

What streams use it?

Inventive Software

28th February 2009, 02:25

What streams use it?

Please tell me that's a trick question. I'd say "stuff I download", but it'd get me struck for rule 6. :rolleyes:

So... almost anything I create with x264?

I ask because it's the standard container for it. After all, MKV was added and that's a non-standard container, despite its popularity.

I guess a sample's needed, yesno?

Guest

28th February 2009, 04:32

I ask because it's the standard container for it. After all, MKV was added and that's a non-standard container, despite its popularity. MKV has a standard. I don't see a lot of MP4 action, so that's why I asked. However, now with Microsoft pushing the segmented MP4 for streaming, it may become more pressing.

I can make my own samples.

aaar9800

8th March 2009, 04:37

Hi neuron2,

I tried to merge 37 AVCHD .m2ts files from my canon hf100 camcorder to then index them together with DGAVCIndex.

When I merge them with either tsMuxeR or tsdoctor or pretty much anything else, loading the .dga through avisynth in virtualdubmod shows that it is 55063 frames, with duration reported as 30:37.26. The demuxed audio is 30:37.896

However, when I merge the files with PIXELA ImageMixer 3 (bundled software with the camcorder), the resulting .m2ts file is indexed to report 55077 frames (30:37.73). The audio is 30:37.824, which is good considering the -66ms delay (I guess this comes from the dropped first 2 frames).

When looking through the generated .dgas, on some of the transitions between two different files, the last gop is sometimes broken off in the PIXELA file, with the last frame of that gop reporting as an IDR. In tsMuxeR's file, the last gop is never broken off and it continuous to be a part of the corresponding "sub"-file. In this case the last frame of the gop(which happens to be a key frame) is not part of the index, and av grows out of sync.

I would have no trouble using ImageMixer 3, if it didn't always crash, wasn't so bloated and slow, and didn't require me to unregister Haali Media Splitter every time I needed to use it.

I was wondering if this inconsistency is a result of the way the other programs merge the files, or if it is fixable within DGAVCIndex. If you want, I could send you both .dgas for the different files, or I could try to reproduce the issue with smaller files and send those.

Thanks for all of your great tools.

Guest

8th March 2009, 05:47

Merge them using DOS: COPY /B.

aaar9800

8th March 2009, 17:26

Merge them using DOS: COPY /B.

That produced an identical file to what tsdoctor made and the 14 frames are still missing.

Guest

8th March 2009, 17:31

I'm not sure what you are asking from me.

Are you saying there is some AV sync issue when using the DOS-joined stream?

aaar9800

8th March 2009, 17:36

Yes, the audio of the DOS-joined stream ends up 467 ms longer.

moviefan

8th March 2009, 17:37

I have not dealt with such a situation, but doesn't DGAVCIndex support the functionality to add multiple files whose order can be adjusted if necessary? Thus you could add your m2ts-files, reorder them to the correct order and create the dga-file? Or is there a reason not to do so? (probably a question particularly for neuron2)

Guest

8th March 2009, 20:02

Yes, the audio of the DOS-joined stream ends up 467 ms longer. I didn't ask about the lengths. I asked if there is an AV sync problem. There are quite valid reasons for a length discrepancy. It's only an issue if there is an AV sync problem.

If there is an AV sync problem, is it a fixed offset or does it grow larger as the stream is played?

@moviefan

Multiple file open is not currently supported. In any case, it would be equivalent to the DOS COPY operation.

aaar9800

8th March 2009, 20:48

Yes, there is an AV sync problem. Starts off fine and then tracks get progressively out of sync.

Turtleggjp

9th March 2009, 17:26

Yes, this is definitely a problem. I experience the same thing with my video camera footage (also HF100). The reason behind this is simple: Video frames go by at 29.97 fps, while audio frames (Dolby Digital in this case) go by at 31.25 fps. It is pretty much impossible for the audio and video to have the exact same duration for all your individual clips, so what happens is that the audio ends up being slightly longer than the video (by a few ms per clip). After joining several clips together, the audio gets progressively more out of sync.

The solution to this problem is simple, and perhaps Neuron2 could add this ability (as an option) into the program. All that needs to happen is to count the number of audio and video frames that go by with each file. The total duration of both audio and video can be calculated from this, and when the audio difference exceeds 16ms (half a frame), the program can simply drop one frame of audio from the output stream, thus now putting it slightly behind. This is in effect what I do with all my footage.

I use DGAVCIndex to index all my files individually (using the command line interface for batch processing), and then use the program AC3 Cutter to examine all the resulting audio files, counting their frames. This series of numbers is then fed into an Excel spreadsheet that I made up, which then determines where audio frames need to be dropped. I then use tsMuxeR to create a single (out of sync) .AC3 file from all my clips, then use the data from my Excel spreadsheet to cut that file using AC3 cutter. The result is a synced (+/- 16ms) .AC3 file that has not been re-encoded.

If this sounds complicated, it kind of is, at least my way of doing it is. The point is, this proceedure does work, and it would be nice if either tsMuxeR or DGAVCIndex could do this instead. Does this sound feasible to add to DGAVCIndex?

Matt

Guest

9th March 2009, 17:45

SeeMoreDigital

9th March 2009, 19:27

Hi Donald,

I've been meaning to ask you for a while.....

Is there any particular reason why the "Frame Type" displays "Not Yet"?

EDIT: Bummer... I've just found a post (for the release of v1.0.1), where you mention the "Frame Type" function has been disabled :o

rebkell

9th March 2009, 19:35

Turtleggjp

9th March 2009, 19:57

The problem is not with the audio rate, it's with the fact that the video granularity is one video frame=33.367 ms while the audio granularity is one audio frame=32 ms. So the clips will not have the same audio and video length if only complete frames are included. The difference then accumulates and the total error will depend on the number of clips.

That's pretty much what I said, except that you were talking about seconds (ms actually) per frame, and I was talking about frames per second. :)

Can't you ask your camera to capture in one file?

The camera creates a new file every time you start and stop recording. If you record for long enough to make a 2GB file (about 17 minutes if I remember), it will then create a new file due to an apparent 2GB file size limit. When it does this though, these files can be joined by a simple DOS COPY /B without causing sync loss.

I don't really see this as a job for a decoder.

A decoder, no. But as something that is demuxing the audio track, it would be nice if it could. I know DGAVCIndex is already counting video frames, since that is what I use it for. Would it be very difficult for it to count the audio frames too? Even if you could have this option for dolby digital tracks only (since that's what most AVCHD camcorders record) it would be very helpful.

Guest

11th March 2009, 01:58

1. Fix a bug in backward GOP stepping.

2. Fixed a problem with Load Project.

3. Added option "Display HD Full Sized".

4. Fixed a problem in M2TS file parsing.

http://neuron2.net/dgavcdec/dgavcdec.html

Sagekilla

12th March 2009, 04:17

neuron2, is MKV support available at the moment?

I'd like to try indexing a Blu-ray movie I ripped using MakeMKV (It's the original m2ts muxed into mkv). I can't copy the movie through other methods (Like DumpHD + DumpVID + aacskeys) because they don't work for me, so I used this.

poisondeathray

12th March 2009, 05:27

is MKV support available at the moment?

Current Limitations
1. Only AVC/H.264 elementary (raw) and transport streams can be opened (no MKV or MP4 files yet).

Guest

12th March 2009, 06:17

I'd like to try indexing a Blu-ray movie I ripped using MakeMKV Demux the streams from the MKV and then process the elementary video stream in DGAVCDec.

Sagekilla

12th March 2009, 06:38

I was hoping I wouldn't have to do that since demuxing a 24 GB file is going to take a while. I suppose it's only thing I can do though.

Any idea if mkv will be supported in the near future?

JK1974

12th March 2009, 12:44

The solution to this problem is simple, and perhaps Neuron2 could add this ability (as an option) into the program. All that needs to happen is to count the number of audio and video frames that go by with each file. The total duration of both audio and video can be calculated from this, and when the audio difference exceeds 16ms (half a frame), the program can simply drop one frame of audio from the output stream, thus now putting it slightly behind. This is in effect what I do with all my footage.

I use DGAVCIndex to index all my files individually (using the command line interface for batch processing), and then use the program AC3 Cutter to examine all the resulting audio files, counting their frames. This series of numbers is then fed into an Excel spreadsheet that I made up, which then determines where audio frames need to be dropped. I then use tsMuxeR to create a single (out of sync) .AC3 file from all my clips, then use the data from my Excel spreadsheet to cut that file using AC3 cutter. The result is a synced (+/- 16ms) .AC3 file that has not been re-encoded.

If this sounds complicated, it kind of is, at least my way of doing it is. The point is, this proceedure does work, and it would be nice if either tsMuxeR or DGAVCIndex could do this instead. Does this sound feasible to add to DGAVCIndex

It seems to be more a problem of TS concatenation that makes problems here.
Have you already tried xport on the concatenated file for demuxing? Similar to ProjectX, this seems to take care of the individual timestamps, so in theory, everything should be kept in sync.

I would also be interested in a solution as I am also a HF100 user - only in PAL land. Haven´t tried before if it happens here also...

buzzqw

19th March 2009, 11:36

i have a little problem decoding this stream

http://www.64k.it/andres/data/Varie/test_dgavcindex.h264 (30mb)

http://www.64k.it/andres/data/Varie/dgavcindex_test.jpg

any help is appreciated

BHH

19th March 2009, 12:32

Looks like the old PAFF problem in the libavcodec version used in DGAVCDec. Not much you can do about it other than switch to DGAVCDecNV or fix the frame-accuracy problems that neuron2 is having with the current libavcodec.

buzzqw

19th March 2009, 13:39

thanks nm

i own the license for DGAVCDecNV but users of my application no (and i suggest to buy it! :) )

thanks again

BHH

Kurtnoise

20th March 2009, 02:41

Don, speaking on this...

The libavcodec.dll shipped with DGAVCDec is a GPL binary. I built it from the
source code available here:

http://sourceforge.net/project/downloading.php?groupname=ffdshow-tryout

I have made a small modification to return a different version string to
manage compatibility issues. This modification is available on request.
can we have access to this mod, please ?

:thanks:

Guest

20th March 2009, 02:44

Already posted. Use search.

Kurtnoise

20th March 2009, 02:52

aaa, got it (http://neuron2.net/misc/ffmpeg1837dg.zip)...thanks.

Guest

20th March 2009, 03:02

You found it faster than me. I just found it. :stupid:

laserfan

26th March 2009, 20:45

My new Q6600 computer is so fast that I've lately found myself watching it perform, and thought I might ask neuron2: is it normal, having done a Save Project on a h.264 es, that DGAVCIndexNV (and non-NV too AFAICT) will scream along for some thousands of frames, and then start behaving in a staccato fashion? Where it races along doing 300-400 frames in a second, then pause for maybe a half-second, then suddenly another 300-400 frames, etc. etc. until completion.

I thought it might be pausing to calculate bitrates perhaps but then as I watch it, that doesn't seem like that's what it's doing. And to look at Windows CPU utilization it does appear that it goes to zero, then 10%, zero, 10% etc. In the end I'm getting an average of 150-170fps.

Not a problem--just wondering it this is normal/expected behavior....? Guess I should go-back & try my old P4 again; maybe it's always worked this same way.

Rodger

26th March 2009, 21:39

You are making a HUGE thinking error.

frames is not equal size!
The reading speed is limited by the bitrate / mass of data to be read.
Parts with higher bitrate will take longer than those with lower bitrate. As I have a Q9400s CPU @3,2Ghz I can tell, that even my super high speed WD Velociraptor-Raid is by FAR not able to feed the Intel-CPU quickly enough with data ;)
That was already the case back then, when I used a C2D E8400@3,55Ghz.

Only thing I can imagine is, that higher cache usage of the program itself might improve speed.
But I have something in mind, that this was already tested some time ago. Well possibly newer systems require different settings.
Usually...reading the data in 1024Kb block sizes is already pretty perfect for most drives. And personally I don´t think huge cache sizes of 8Mb and more will do any trick to it.

@ Neuron: If you give out a test-version with variable cache sizes, I´d be helpful to test the perfect settings for newer quad core machines.

@ laserfan: Try out An Intel X25-M SSD , this surely will help speeding up :D

Guest

26th March 2009, 21:59

I don't know and I'm not that interested to be honest. :)

I have a large buffer that gets refilled. Trying to tweak it is not interesting to me given all the other things on my list already.

But never say never.

laserfan

26th March 2009, 23:38

Honestly my question was not re: improving speed--I was just wondering if the program's zoom-stop-zoom-stop behavior was normal. And I guess you have said "yes", or maybe you have just never noticed! :)

No matter then; must be "stopping" when the buffer fills. And it "screams along at the beginning" maybe cuz there's mostly black there at first. And I managed to post this accidently in the non-NV thread; tho AFAICT both versions work the same... :slinksaway:

Guest

26th March 2009, 23:48

No need to slink. The start stop thing I've only seen on the NV builds. It never got high enough on my list to look at. I do know that the NV builds have a much bigger buffer to refill. Maybe after I merge them all, add program stream support to DGMPG, support multiple file loading. It's not broken but it could possibly be improved.

Egh

13th April 2009, 04:41

@neuron2:

I have rather interesting case here with one m2ts file from bluray source. It has video stream which is fine, but also two other audio streams detected (LPCM 48KHz 2ch), however if I demux them they are empty. Length is normal but streams are filled with zeros :) Splitters also have troubles with that m2ts, for instance Haali splitter doesn't detect audio streams at all, apparently. Any suggestions re what experiments I can do with the file to identify what can be wrong with it?

Revgen

13th April 2009, 20:37

^You might want to post a sample for neuron2 to look at.

Guest

13th April 2009, 20:56

Yes, indeed. Please post a link to an unprocessed source sample that can be used to duplicate your issue.

Egh

14th April 2009, 18:38

Try this, 25mb cut from beginning. http://www.mediafire.com/?z4kzngyodty

Maybe this video doesn't really have any sound data, but both VLC and Haali splitter have troubles with it regarding these two additional tracks. VLC considers them to be video (???) and Haali doesn't detect them at all apparently.

Guest

14th April 2009, 20:10

It seems to be just blank audio going with some kind of intro video. Is there audio present when you play this section of the BD?

Egh

14th April 2009, 21:14

It seems to be just blank audio going with some kind of intro video. Is there audio present when you play this section of the BD?

I cannot play it :shy: This is separate intro yes, it is not in the main m2ts file. And yes, this file raised suspicions because both Haali and VLC have troubles with additional tracks. Still puzzled why two blank audiotracks were added to that file.... Main m2ts file contains two audiotracks as well, but they are not blank....