DGIndex 1.3.0 beta 5

[Guest]

OK guys, here's the prime-time video demuxing. It will start saving at the first sequence header for maximum compatibility with other applications. That means if you have stray frames at the beginning before the first sequence header, you'll lose them, which will affect audio sync. Valid M2V files need to have a sequence header before any pictures.

The output file is in the same directory as the input files and takes the name of the first file but with extension changed to "m2v".

http://neuron2.net/dgmpgdec/dgmpgdec130b5.zip

[Zep]

Originally posted by neuron2
OK guys, here's the prime-time video demuxing. It will start saving at the first sequence header for maximum compatibility with other applications. That means if you have stray frames at the beginning before the first sequence header, you'll lose them, which will affect audio sync. Valid M2V files need to have a sequence header before any pictures.

The output file is in the same directory as the input files and takes the name of the first file but with extension changed to "m2v".

http://neuron2.net/dgmpgdec/DGIndex130b1.zip

straight demux or more like projectx where it fixes audio sync
and also inserts silence in the inserted audio chunks to get sync,
at an example, in drops of a HDTV .ts stream.

sync at each GOP? each frame?


is there a way to detect dupe frames at this level and then
when making the d2v just write all the good frames out. Does
a .ts stream even have that info or are we stuck always having
to decimate? i ask because my HDTV card does in fact decimate
in real time to the screen but the .ts file has all the dupes
of course. So it is does appear that it is possible to read the
ts and know what to decimate but my card could be doing some
hack to do that and not be following the standards/true specs.

[TEB]

Neuron2: Ive tested some problematic files on it and im seeing some strange things. Plz dl the new u5712005 file and check what u can see. Im getting macroblocking and various video errors and our test eq. says the file are perfectly fine. (mprobe, matzasina mpegts analyzer)

I have another file to: a invalid one i think (made with some optibase eq.. sucky..) its an mpeg1 system file with mpeg2 ES streams within.. DGindex just crashes or times out. Ive put it on the server to for u to test.. DGindex should spit out some kind of warning popup. File outa bounds or something non mpeg spec.. = emil_lonneberget.mpg

best regards te

[bond]

i didnt test it but is video demuxing supposed to work with mpeg-4 (asp and avc streams) in .mpg?
what about aac btw?

[SeeMoreDigital]

Originally posted by bond
i didnt test it but is video demuxing supposed to work with mpeg-4 (asp and avc streams) in .mpg?
what about aac btw? Might be worth a try!

I've just fed it with an Mpeg2 .TS file with AAC audio. And after manually entering the PID information, the file was de-muxed perfectly.

Many, many thanks for this tool DG :D


EDIT: Hi bond...

Just tried loading an MainConcept H264/AVC .mpg file into it.

And although DGIndex 1.3.0 beta1 swallowed the file, I was not able de-mux the streams. Sadly it was not possible for me to gain access to the files PID data, via the "Stream" option setting :(


Cheers

[ppera2]

Hmmm... folks...
Why immediatelly starting with big expectations?
Can it Mp4, AAC... ?

I think that programmer will be enough busy with fixing problems by Mpeg2 demuxing in next period.

Knowing Donald, I'm positive about new features implementations in future :)

[bond]

Originally posted by SeeMoreDigital
Just tried loading an MainConcept H264/AVC .mpg file into it.

And although DGIndex 1.3.0 beta1 swallowed the file, I was not able de-mux the streams. Sadly it was not possible for me to gain access to the files PID data, via the "Stream" option settingmainconcept avc .mpg transportstreams are b0rked

[SeeMoreDigital]

Originally posted by bond
mainconcept avc .mpg transportstreams are b0rked Do you have any other samples I could try?


Cheers

[Guest]

DGMPGDec is an MPEG2 decoder. It demuxes M2V video. DGIndex has for some time been able to demux AAC audio.

There is currently no special handling for damaged streams. I plan to write a preprocessor for fixing up damaged streams.

I added this feature because I needed to be able to make a single M2V from a series of VOBs. If it is useful for anything else, that is gravy to me.

I will have a look at TEB's u5712005 file.

[SeeMoreDigital]

Hi Donald,

How difficult would it be to add Mpeg1 support?


Cheers

[Guest]

Originally posted by SeeMoreDigital
How difficult would it be to add Mpeg1 support?
Funny you should ask. That was next on my list. I don't expect it to be very difficult.

[Guest]

Originally posted by TEB
Plz dl the new u5712005 file and check what u can see. Im getting macroblocking and various video errors and our test eq. says the file are perfectly fine. I loaded the transport file and did F5. I saw numerous video errors (1 and 2) but I saw no macroblocking. I then demuxed the video, and loaded that into DGIndex and did F5. It gave exactly the same result: video errors at the same places and no macroblocking. The problem is in your source. DGMPGDec is pretty good at recovering from these errors now. If you import the M2V into another application that is less capable in that regard, you may have problems. The whole area of repair for transport streams is something I am looking at.

EDIT: The latest BSPlayer plays the demuxed M2V just fine, no sign of any macroblocking.

I have another file to: a invalid one i think (made with some optibase eq.. sucky..) its an mpeg1 system file with mpeg2 ES streams within.. DGindex just crashes or times out. Ive put it on the server to for u to test.. DGindex should spit out some kind of warning popup. File outa bounds or something non mpeg spec.. = emil_lonneberget.mpg I'll look at this tonight.

[TEB]

strange.. I get some macroblocking when i have some heavy field"ing" during the company logo in the beginning..

Check ure pm for a log from the mpeg2 analyzer and see if u can understand why some PRO analyzers say its ok and dgindex says otherwize..

[Guest]

Originally posted by TEB
strange.. I get some macroblocking when i have some heavy field"ing" during the company logo in the beginning.. OK, I stepped through and did see a bad frame or two during the Universal logo. You've got bad video data, what can I do?

Check ure pm for a log from the mpeg2 analyzer and see if u can understand why some PRO analyzers say its ok and dgindex says otherwize.. [/B] It's a stream analyzer, right? Does it decode every video slice? I doubt it.

[Cyberia]

Don, this version fixes the demuxing problem I had before with the FTE6.vob. Good job.

EDIT: However... You have a problem with an error dialog that appears if the m2v file cannot be opened. It just keeps retrying and retrying forever.

Save a project, open the m2v in vdubmod and try to resave the project.

[Guest]

@Cyberia

Yes, that's right. It catches you in a very long loop. I wanted to punish the user. But you've convinced me that it is not right to abuse my users like that. Sorry.

Here is a kinder, gentler version. The user will need to seek salvation elsewhere.

http://neuron2.net/dgmpgdec/DGIndex130b2.zip

And, oh, it didn't retry forever. It only retried for each sequence header. Please try to be more accurate with your trouble reports in the future. We're counting on you! :D

[Cyberia]

Well no doubt the users deserve it! You are a merciful programmer indeed.

This version now aborts the m2v creation in this instance, and goes ahead and writes the d2v and ac3 files. Not that thats wrong... I just expected it to abort the whole shebang. You exceeded my expectations.

Ummm... On a hunch though I found a new bug. If the audio file (AC3 in this case) is in use when you try to save a project, DGIndex CRASHES. oh the agony.

And, what the heck. Here's a third odd thing. DGIndex doesn't lock the d2v file when it's writing it out. You can open two instances of DGIndex and have them both write to the same d2v at the same time. Dooesn't crash though!

[Guest]

The horror.

I'll never be the same again.

When I have hunches, I call them Grunches = Graft hunches. In your case I suppose they could be Crunchies.

[Cyberia]

Files do not have to be in use, setting read only and re-saving project does same effect.

FYI (some more crunchies) :

LPCM (Decoded to WAV) ... same problem, crashes DGIndex if it can't write the file. I assume all audio types have same problem.

AVS scripts (automatic from templates) ... does NOT crash DGIndex if it can't write the file. Does not write the file, but does NOT complain that it can't write the file either. (this is minor, but now you know :) )

[Cyberia]

And another thing...

How the heck do you manage to get 1700+ views on this thread in TWO DAYS!!! You're just about to exceed the FAQ views accumulated over two MONTHS.

Some guys have all the luck...

[len0x]

Originally posted by Cyberia
How the heck do you manage to get 1700+ views on this thread in TWO DAYS!!!

The thread was linked from front page news on doom9.net

[Wilbert]

Graft, is this pc_scale/tv_scale supposed to work with VFAPI Reader Codec v1.05?

It doesn't do anything, that's why I ask. If I create a m2v with Y=16,U=V=128, two d2v from it using pc_scale and tv_scale, two fake avis from those, both clips have R=G=B=0.

[Guest]

I mentioned in the other thread that VFAPI forces TV scale RGB.

[Wilbert]

You mean PC scale (cause it does YUV [16,235] -> RGB [0,255])?

[Guest]

Correct, sorry.

[Trahald]

is it possible to have the save file use the .d2v filename .. i loaded a m2v file with the idea of cutting it down via the demux option, but it overwrote my original. im assuming since it was intended for files with .vob extension as source?

[D3s7]

I had this issue as well..

Opened a .m2v file, set my input/output range and did "save project and demuxed"

it created a .d2v file and killed my .m2v... (was left with a 1k file that wasn't even the clip of the original)

This was with saving into a different directory as well.. not same as original

[Guest]

OK, I never thought people might use it to cut elementary streams, but why not? I'll fix it to use a different filename in that case. Thank you for bringing it to my attention.

[Cyberia]

Originally posted by Trahald
is it possible to have the save file use the .d2v filename .. i loaded a m2v file with the idea of cutting it down via the demux option, but it overwrote my original. im assuming since it was intended for files with .vob extension as source?

Oops. I had intended to mention this to Don but I forgot.

The m2v file uses the name of the original vob, but the audio files and the avs script use the name of the d2v project.

For consistency, I recommend that the m2v file use the project name as well, however..... please default the project name to the name of the original vob.

Currently there is no default project name provided when saving.

[Cyberia]

Originally posted by D3s7
I had this issue as well..

Opened a .m2v file, set my input/output range and did "save project and demuxed"

it created a .d2v file and killed my .m2v... (was left with a 1k file that wasn't even the clip of the original)

This was with saving into a different directory as well.. not same as original

In this case, we should have simply generated an error dialog saying "Cannot write m2v file" and gone on.

We shouldn't be able to overwrite the file we are reading from!

[Guest]

The user could specify a project name (D2V file) that has the same base as the input file, e.g., input Cyberia.m2v and project file Cyberia.d2v, so I have to allow for that.

This is what I will do:

d2v_base.demuxed.m2v

So, in the example, it would be Cyberia.demuxed.m2v.

If the input file is Cyberia.demuxed.m2v, then the output will be Cyberia.demuxed.demuxed.m2v, etc. :)

[Guest]

I posted 1.3.0 beta 3:

http://neuron2.net/dgmpgdec/dgmpgdec130b3.zip

It fixes the M2V overwrite issue.

But watch out! This really is a beta. I made some major changes to EOF handling. If there is not an incomplete picture at the end, the frame counts should be correct, and the last picture should display normally when previewing. I also added exception handling to both DGIndex and DGDecode, so that we don't crash on an incomplete picture.

If there *is* an incomplete picture at the end, you can get an incorrect frame count when previewing in DGIndex.

But finally I have a version that works correctly with my test files that have exactly one, two, and four frames.

[cw_uk]

I have been working with some HDTV transport streams lately did you know that DGIndex will lock up should you select more then 10 files at once when loading your sources.

[Cyberia]

Originally posted by cw_uk
I have been working with some HDTV transport streams lately did you know that DGIndex will lock up should you select more then 10 files at once when loading your sources.

CONFIRMED!

I just checked and both 1.2.1 and 1.3.0b3 lockup (cpu usage goes to 99% and stays) when opening more than 10 vobs at once. ONLY THROUGH THE FILE/OPEN DIALOG. DRAG-AND-DROP SEEMS UNAFFECTED.

It's intermittant at 11 vobs, but consistently happens with 16+ vobs.

[Guest]

I opened 20 by loading the same 5 four times. Nothing wrong happened. Does that fail for you, or do I have to open all 20 at once?

[Cyberia]

You have to open them all at once via the File/Open dialog.

Follow these steps:

Close all DGIndexes.
Open DGIndex.
Select File/Open/Add.
(Windows File Select dialog appears.)
Select 16+ vobs at once.
Click Open. --> [lockup 99% cpu usage]
(DGIndex does not return to it's File/Open dialog)

I can reproduce this on two different systems.

Drag-and-Drop seems to work fine for the same set of vobs.

[Guest]

OK, the file dialog buffer that returns the file list must be too small. I'll fix that when I get back from LA. In the meantime, the workaround is to load in bunches.

[Backwoods]

I figured out my problem from the previous threads and your betas. The problem I had was when saving, DGIndex would lock up.

Now it does the same, but if I let it sit for about 20sec the save runs. But before I could let it sit for a minute or longer and it would still not save.

Is there anything that can be done about that? I'm uploading the type of files I use, it could be because of them.

[Cyberia]

Originally posted by Backwoods
I figured out my problem from the previous threads and your betas. The problem I had was when saving, DGIndex would lock up.

Now it does the same, but if I let it sit for about 20sec the save runs. But before I could let it sit for a minute or longer and it would still not save.

Is there anything that can be done about that? I'm uploading the type of files I use, it could be because of them.

What's the file name(s)?

[Backwoods]

backwoods_sample.mpg

I doubt it's my machine...could be, but doubt it. Other MPG2s work fine.

[Cyberia]

Works fine for me once I do a "Detect PIDs: Raw"

Don, is the Info panel supposed to appear on the left side, over the video for TS?

[Guest]

Originally posted by Cyberia
Don, is the Info panel supposed to appear on the left side, over the video for TS? Yes. It's determined by the displayed picture size.

[Backwoods]

Originally posted by Cyberia
Works fine for me once I do a "Detect PIDs: Raw"

The video comes up fine for me, it's when I goto save the D2V. There is a pause. You do not experience that with this clip?

[Cyberia]

Originally posted by Backwoods
The video comes up fine for me, it's when I goto save the D2V. There is a pause. You do not experience that with this clip?

nope

EDIT: How fast is your cpu? I see a very brief (1 second) delay where the Info panel is completely unpainted. Just a big grey box. Then the fields fill in and everything is OK.

Also: What other settings are you using? What PIDs, what video and audio settings?

[Backwoods]

2.8 HT w/1gig ram.

0x810=v
0x814=a

32bit SSE2 MMX

Thats the box I recieve too. Greyed out, then after 20 seconds or so (longer in previous betas) the project saves.

[Guest]

@Backwoods

I duplicated this on my laptop. But I can't fix it until I get back to Chicago on the 19th.

The problem is that your stream has no PAT/PMT and when you save project, DGIndex is trying to parse the PAT/PMT to get the audio type. It reads through the entire file. It's an easy fix but I'm away from my 1.3.0b3 source code and I don't want to branch off an earlier version.

[Backwoods]

Glad to hear you found the cause. The footage is from the Sony HDR-FX1 HDV camera. So maybe it has something to do with how the camera creates MPGs or the program transfering the footage? Currently I am using CapDVHS.

[Guest]

Oh, then it's not surprising that there is no PAT/PMT.

BTW, your bird is cool; my Peekoo (a dusky conure) would be very impressed.

How much does that camera cost?

[Cyberia]

Is there some reason why I am unable to duplicate the extensive delay?

[squid_80]

I get the same thing when processing transport streams recorded with the software that came with my card - a FusionHDTV DVB-T Lite. Never thought of it as a bug, I just presumed it was normal for .ts files.