What exactly does DGIndex do?

[bur]

I always wondered why exactly a tool like DGIndex is neccessary. I only found information like "it indexes the VOB files".

But what exactly does that mean? Does a VOB have a special structure that makes it impossible to directly access a specific frame? And what does the D2V file contain, a list that says "frame 1352 can be found at offset 1020"?

[LoRd_MuldeR]

In contrast to, for example, AVI files, MPEG-TS and MPEG-PS files (which includes VOB files) do not contain a "global" index block.

However if you want to implement frame-accurate random access, which is required for video editing (e.g. Avisynth), then you will ultimately need some kind of index.

And that's what DGIndex creates. FFmpegSource basically does the same thing, when you open a file for the first time. So does the VirtualDub MPEG-2 plug-in.

The index stores the exact file position for each frame number. Actually it will also have to store the frame type, as you cannot start decoding at an arbitrary frame (only at I-Frames).

Without an index, you'd have to start decoding at the very beginning of the file and process all frames until the desired frame number is reached. That would be SLOW.

[CarlEdman]

Lord_Mulder as usual is quite right.

But what I have always wondered why in applications where by definition we need to decode all the frames in (roughly) sequential order anyway, like transcoding, there isn't some more efficient way that skips the unnecessary indexing.

[LoRd_MuldeR]

If you just play back the file, then you don't need frame-accurate seeking.

Instead, for playback, it is perfectly okay to "blindly" jump to an arbitrary position in the file and, from there, start decoding (playback) at the first I-Frame you'll encounter.

Also, for simple transcoding tasks, where you just read the complete input file from beginning all through the end, you don't need to worry about seeking at all.

Non-linear video editing, like Avisynth does, is a completely different thing! You need to be able to seek to exact frame numbers. And you need to be able to do this in a "fast" way.

You can't "jump" to the correct position without extra info, because it simply is impossible to know beforehand where exactly frame #n is stored in the file.

Also, restarting from the very beginning of the file on every single seek and counting frames until the desired one is reached, is not an option either - this would be much too slow!

Parsing through the entire file and building an index takes some time, yes. But you have to do this only one time! And, once you have the index, seeking is maximum fast.

[Blue_MiSfit]

Correct. DGIndex (more properly speaking, DGDecode) also contains a complete MPEG-2 video decoder, which supports almost anything you can throw at it. This is a very good thing!

Derek

[vampiredom]

DGIndex (more properly speaking, DGDecode) also contains a complete MPEG-2 video decoder, which supports almost anything you can throw at it. This is a very good thing!

It is, indeed. I have done a fair amount of testing with DGIndex/DGDecode and all its variants (AVC, NV, DiAVC...). These tools are just great.

It really does not take very long to index such files on a fast computer.

My issue with such indexing was not its speed but inconvenience, as I saw it, for having to open and index each file and then deal with the audio formats and such. Granted, DGIndex does allow for using a script template but I didn't feel this provided me with the flexibility I wanted (such as the ability to use other batch scripting tools, etc.)

I wrote a little "helper app" called MpegAutoIndex and a function, MpegAutoIndexSource() to automate this process for me. Perhaps you may find it useful as well.

http://forum.doom9.org/showthread.php?t=162930

This is what I use whenever I have an MPEG2 or MPEG4 source I need to open with AviSynth. It allows one-step sourcing of these files (ala FFMPEGSource) with the reliability of DGIndex/DGDecode, etc.

[Blue_MiSfit]

There are ways of doing frame accurate access without a full indexing pass. Some "pro" apps like Carbon Coder have support for frame level trimming and concatenation for long GOP content during a transcode, without any special indexing, which is extremely useful.

I've had pretty good results with DirectShowSource combined with the Gabest MPEG-2 splitter, and ffdshow's libavcodec MPEG-2 decoder. Take that with a grain of salt, but it's worked okay for me. Granted, I don't do a lot of trims ;)

Derek

[Wilbert]

My issue with such indexing was not its speed but inconvenience, as I saw it, for having to open and index each file and then deal with the audio formats and such. Granted, DGIndex does allow for using a script template but I didn't feel this provided me with the flexibility I wanted (such as the ability to use other batch scripting tools, etc.)

I wrote a little "helper app" called ...
Could you stop spamming our forum with your tool? It's great that you have made a nice addition to help automate processes, but there is no need to bring it up on every occasion. It has nothing to do with the topic of this thread nor many other thread where you bring it up.

[vampiredom]

Could you stop spamming our forum with your tool?
Sorry. I certainly do not intend to "spam". I apologize for coming across that way.