DGMPGDec 1.4.6 Final

[Guest]

Here is 1.4.6 Final.

http://neuron2.net/dgmpgdec/dgmpgdec146.zip

1. Fixed a bug in indexing for some field structure streams.

2. The 'This I picture starts a new GOP' flag in the D2V file was not being
set correctly. This flag is not used by DGDecode, so the impact was only to
the diagnostics produced by Parse D2V.

3. The D2V file format version number was bumped to 12. (This should have
been done for version 1.4.5; better late than never.)

4. Fixed a problem in DGDecode that could cause a lockup when navigating
on the timeline with some unusual field structure streams.

5. Increased the maximum number of input files from 256 to 512.

6. Incorporated fix to DGVfapi (by 'tritical') that prevents a failure when
a large number of VFAPI-served files are opened at once.

7. Changed DGIndex/DGDecode so that when DGDecode sets the frame rate
in Avisynth, it uses the exact fractional representation for
fps_numerator/fps_denominator (changes by 'tritical').

This way, e.g., NTSC clips get the exact 24000/1001, 30000/1001, or
60000/1001 rate instead of 23.976, 29.970, or 59.94.

DGIndex was actually getting the framerate correct, but when it wrote the
d2v file, it would multiply the floating point value up by 1000 and then save
it. Then DGDecode would use that value for the numerator and 1000 for the
denominator. Now, when DGIndex gets the fps value from the lookup table it
also gets numerator/denominator values from a lookup table. Then in the d2v
file it writes, e.g.:

Frame_Rate=29970 (30000/1001)

with numerator and denominator in parentheses. DGDecode then uses those
values when setting the frame rate in Avisynth.

This change bumps the D2V file format version number to 13.

8. The message box that pops up for transport streams when no data is
found has been clarified to direct the user to use the Stream menu to set
the PIDs.

9. Fixed a hang when the CLI -HIDE option is used and the input file cannot
be opened.

10. The video overlay is now not claimed when the -HIDE or -MINIMIZE CLI
options are used.

11. File version properties were added for DGIndex, DGDecode, and DGVfapi.
If you right click on the EXE, DLL, or VFP files and select properties, you can
now obtain file version information.

12. The DGIndex Information dialog now shows an overall average bitrate for
the entire previewed video in addition to the windowed value. The windowed
value now reports every 32 frames instead of every 64 frames.

13. DGVfapi's handling of audio from avisynth scripts was fixed. It didn't set
values for dwBlockAlign and dwReadedSampleCount, and it should be
converting the output to 16-bit integer if it isn't already in that format.
Changes by 'tritical'.

14. The old "Fix D2V" tool is back in a new guise: the "Correct Field Order"
option. Refer to the manual for details.

15. Fixed the truncation of the last few bytes of demuxed M2V files.

16. Fixed a problem that limited the number of files that could be opened at
once from the File Open Dialog (drag-and-drop to the DGIndex window was
OK). It should now be possible to open up to 512 files at once via the open
dialog.

[richardlfyang]

Tks! I'll try it! :)

[AllTimeSToneD]

It seems like i discovered accidently a bug (or a limitation)?

If i try to open 314 transport stream files each 100mb in size DGIndex gives me a error window and crashed once clicked OK or Cancel.

Something about "0x00418e82 ..... Memory 0x00000000"

I could post a screenshot of the error window but that probably wouldn't help much as my WinXP is on german. I've tried to limit the number of files to 256 files and it worked. So it looks to me like a limitation of max 256 files.

Is that one fix-able? I would like to be able to open all the files at once.

Btw the bug/limitation exists on both (using drag and drop and open file/add).

[Guest]

MAX_FILE_NUMBER is currently 256, as you suggest. I will bump it to 512 in the next beta. Will that be enough? What is your application scenario that causes such a situation?

[AllTimeSToneD]

Will that be enough?
Yes probably yes :thanks:

What is your application scenario that causes such a situation?

Capping a European HD stream of RotK while the streaming software splits the incoming stream in 100MB chunks. That would result as you can already guess in a few hundred ts files. :D

[pbkg]

Is there any difference between the processing of PAL and NTSC in the program? If not then I guess you could ignore this, but otherwise, there may be a problem...

Why? Looking through the source code of previous versions, it seems that unless framerate == 25 fps, then the preview gives the video type as NTSC when it should be PAL. But, the two Australian government owned broadcasters both have decided that their "High Definition" (snigger, snigger), service should be 50 fps, rather than the standard 25 fps (two of the commercial stations run their HD at 1080i PAL @ 25 fps as far as I can tell, and the other is all over the place like a dogs breakfast for me, with their "HD" service PID currently the same as the SD PID, ie the same video, but different audio....). As far as I can tell, this is part of the digital standard (50fps via PAL @ 720x576), but it could catch people out. I would upload a copy of a file, but I would be here all night on my connection....

[Guest]

That is only a display issue for the info dialog. Thanks for pointing it out. Maybe it's worth displaying 50fps as "PAL".

[Tima]

As far as I can see, there's no explanation in help files about what 'Video %xx' in 'Video Type' means.. 'Bug'? :)

[foxyshadis]

As far as I can see, there's no explanation in help files about what 'Video %xx' in 'Video Type' means.. 'Bug'? :)
http://neuron2.net/dgmpgdec/DGIndexManual.html#FieldOp

This is the same help file that comes with DGIndex. (Although you sort of have to know what you're looking for to look for this chapter.)

[Guest]

Actually, Tima found a document bug, because I still had "FILM" vs. "NTSC", although the display now shows "Film" vs. "Video". Here are the corrected sections:

http://neuron2.net/dgmpgdec/DGIndexManual.html#FieldOp
http://neuron2.net/dgmpgdec/DGIndexManual.html#video

[calinb]

I’m having trouble with HDTV .mpg files created by my Dvico Fusion5 card. Dgdecode doesn't find/demux the AC3 audio track. Dgdecode handles Fusion5 .ts files okay but, for some strange reason, my Fusion5 card creates corrupted .ts files from one QAM channel that I like to watch (PBS) so I must record to .mpg files on this channel. I found that the Windows mpeg demultiplexer DSF works great (audio + video), but the Windows splitter DSF and MPC internal parser/splitter seek very poorly. The Elecard/Moonlight DSF doesn’t work at all! The Fusion playback filters work fine, of course.

Any chance I could interest you in a sample file, Donald? Thanks much, if you have the time to take a look.

[Guest]

Yes, sure. Upload it to my FTP and notify me of the filename. You didn't say what problem you have with DGIndex!

I am involved in another thread about this card using .mpg, in which DGIndex can't get the audio right, but gets the video fine.

Is yours the Australian or ATSC format card?

[calinb]

doh! Sorry Donald, I just realized my ommision and edited my posting, but you replied before I could get it finished. Yeah, it's the audio. I'm posting to the Dvico Q/A site about the corrupted .ts files on this one cable 256 QAM channel too, but the card should create either .mpg or .ts (.tp) output files. Both output options work on all channels but this one, which requires mpg output. I have the ATSC card. Point me to the other thread and maybe I can help there too. They could try transport stream output, as an alternative.

[AMED]

I have a small request with regarding saving a project file.

Could the default behaviour be changed to check the name of the parent folder and if it isn't named VIDEO_TS then use that name of the parent folder for the project name?

eg.
if parent folder name is not VIDEO_TS
use parent folder name for naming of project when saving
else parent folder name is VIDEO_TS or files are from multiple locations
use current naming convention
end

[Guest]

doh! Sorry Donald, I just realized my ommision and edited my posting, but you replied before I could get it finished. Yeah, it's the audio. I'm posting to the Dvico Q/A site about the corrupted .ts files on this one cable 256 QAM channel too, but the card should create either .mpg or .ts (.tp) output files. Both output options work on all channels but this one, which requires mpg output. I have the ATSC card. Point me to the other thread and maybe I can help there too. They could try transport stream output, as an alternative. http://forum.doom9.org/showthread.php?t=101507

It appears to be a very similar, if not identical case.

I don't see a file uploaded from you yet.

[Guest]

Could the default behaviour be changed to check the name of the parent folder and if it isn't named VIDEO_TS then use that name of the parent folder for the project name? Sorry, but no. Different people have different conventions for where they save project files, and if I try to implement them all, they'd conflict. Better to keep it simple and basic.

[calinb]

http://forum.doom9.org/showthread.php?t=101507

It appears to be a very similar, if not identical case.

I don't see a file uploaded from you yet.
Sorry about the delay in uploading the samples, Donald. I uploaded calinb_cap.zip to the site. I also sent the zip file to Dvico (Fusion developer). It contains an .mpg, .tp, and .m2t file.

Fusion crashes after many seconds or a few minutes with .tp capture only. The .tp files are usually corrupted. Dvico is working that issue.

The .tp and .m2t files may or may not be useful to you in getting to the bottom of this. The .m2t file was captured from the firewire port of my cable STB on the same channel. with my Apple Mac mini. Again, may or may not be related to gumballguy's and my problem. If you can make DGIndex work with the .mpg file, we could abandon other cumbersome workarounds!

Donald, please check my PM for a Dvico contact address. Again, may or may not be useful.
:thanks:

[mg262]

In the MPEG2Source documentation with this release, there is the following line:

0: Use value specified by DGIndex (iDCTs 6 and 7 unavailable in DGIndex)

I have no problem selecting those modes in DGIndex, and they are documented in its manual. Is it that this line has not been updated yet (which would be entirely understandable given the rate at which you update the software), or -- probably more likely -- am I misreading it completely? If the latter, I would be grateful for the correct reading...

This is a very minor point... I hope you don't mind me bringing it up.

Thanks in advance,
M.

[Guest]

It's a document bug. Read it this way:

0: Use value specified by DGIndex

Thank you for pointing it out. It will be repaired in the next beta.

[Tima]

Donald, how much is the write buffer of DGIngex for now? Could you make it configurable to o just bigger (to make user able to avoid fragmentation of files that are written during demux, when f.e. used vobs are on the same drive)?

[Guest]

Right now I don't call setvbuf() at all, so I just get whatever the default is for fopen(). How big a buffer are you thinking of and do you have a good reason to believe it will actually reduce fragmentation?

What does this mean: "f.e. used vobs"?

[Tima]

Ehm.. bad english, sorry :)

f.e. = e.g.

I don't know about the exact influence of buffer on fragmentation, but I'd like to choose the the size of blocks I'll feed my HDD with. I believe it'll at least improve the performance.. ;)

Two hours ago I had a situation when my two demuхed 384MB and 192MB files had about 3000 fragments each.. I think that buffer should be _at least_ 512KB.. (or configurable in range 512KB - 16MB or maybe even more).

Also _maybe_ it's good to specify read and write buffers separately..

[Guest]

I'll experiment with it. What tool are you using to assess fragmentation of a given file?

[Tima]

I use Raxco PerfectDisk 7.0, but it can de done also with Windows built-in defrag utility.

[Guest]

* Fixed a problem in DGDecode that could cause a lockup when navigating
on the timeline with some unusual field structure streams.

* Increased the maximum number of input files from 256 to 512.

* Incorporated fix to DGVfapi (by 'tritical') that prevents a failure when
a large number of VFAPI-served files are opened at once.

http://neuron2.net/dgmpgdec/dgmpgdec146b2.zip

[SeeMoreDigital]

Hi Donald,

A few days ago I was given a 75MB 1920x1080/88 MPEG-2 .TS sample with DTS audio to try with an high-def network capable DVD player I'm currently beta testing.

While the sample provided appears to play perfectly in the affore mentioned player. When I run the sample thru' DGIndex 1.4.6 beta 2, only the video stream is de-muxed.

I tried manually adusting the PID's (there are only two) but withut success!

Would this e a sample you are interested in? If so I will upload it to your server!


Cheers

[Guest]

Sure, go ahead. Upload it and then PM me with the file name. Thank you.

[Tima]

Donald, how can I tell dgdecode to return all frames, even corrupted?

I have an mpeg-2 file (tv-capture). I make d2v file from it (and demux the audio in mpa file, and getting wav from it) and then put it into Avisynth script:
mpeg2source("E:\!TV Rec\МиМ 5 [NeXT]\МиМ 5 [NeXT].d2v", idct=0).AssumeFrameBased()The last frame's number is 85737.


Then i put that mpeg-2 file in script via DirectShowSource():
DirectShowSource("E:\!TV Rec\МиМ 5.raw.mpg").AssumeFrameBased()The last frame's number is 85771 in this case. In particular, it shows extra frame #0, that is corrupted.

While looking the movie in media player (MPC) the movie plays synchronized.
When i take video from dgdecode, and audio from demuxed file, it's not synchronized in the end - video plays earlier.

My goal is to get synchronized video and audio using only DGMPGDec and avisynth.

[Guest]

The frame count from DGIndex should be accurate. When indexing, the frames are not decoded, so frames will only be lost if key sequence codes are corrupted.

Regarding the synchronization, did you get an offset in the demuxed audio's filename and if so did you correct for that? Is the desync a constant offset or does it grow bigger as the movie progresses?

I regret that I can't look at your material until I return from vacation.

[Tima]

Regarding the synchronization, did you get an offset in the demuxed audio's filename and if so did you correct for that?
Yes, the delay was -131ms, and I corrected it when converting to wav. But in the end there's a misalignment about 1-2 seconds, not +-0.131 :)

Is the desync a constant offset or does it grow bigger as the movie progresses?Can't determine exactly, but it appears that audio shifts somewhere in the end of movie..

[AMED]

I think i've found a bug/quirk in DGMPGDec 1.4.6 beta 2.

Things you need to reproduce
DGMPGDec 1.4.6 beta 2
My computer window

load a movie into DGMPGDec and save the .d2v project, while the slider is moving from left to right and while DGMPGDec is not minimised, right click any folder in the my computer window, and choose properties, click cancel on the properties box, then right click on a blank area in the my computer window, have a look at DGMPGDec and you'll see that is has paused the processing the movie, until you exit that last right click menu.

[Guest]

Can't determine exactly, but it appears that audio shifts somewhere in the end of movie.. DGindex determines the delay only once at the start of the project range. You'll have to fix it manually by dividing the project into two pieces.

[Guest]

I think i've found a bug/quirk in DGMPGDec 1.4.6 beta 2.

Things you need to reproduce
DGMPGDec 1.4.6 beta 2
My computer window

load a movie into DGMPGDec and save the .d2v project, while the slider is moving from left to right and while DGMPGDec is not minimised, right click any folder in the my computer window, and choose properties, click cancel on the properties box, then right click on a blank area in the my computer window, have a look at DGMPGDec and you'll see that is has paused the processing the movie, until you exit that last right click menu. That sounds like a windows quirk, but I'll have a look at it when I return from vacation. Thank you for pointing it out.

[AMED]

That sounds like a windows quirk, but I'll have a look at it when I return from vacation. Thank you for pointing it out.I had a feeling it was that, have a great vacation and a merry christmas.

[Cyberia]

I think i've found a bug/quirk in DGMPGDec 1.4.6 beta 2.

Things you need to reproduce
DGMPGDec 1.4.6 beta 2
My computer window

load a movie into DGMPGDec and save the .d2v project, while the slider is moving from left to right and while DGMPGDec is not minimised, right click any folder in the my computer window, and choose properties, click cancel on the properties box, then right click on a blank area in the my computer window, have a look at DGMPGDec and you'll see that is has paused the processing the movie, until you exit that last right click menu.


I'll look into this this weekend.

[CEC]

I found this line in the "DGDecodeManual.html":

"All of the available options are IEEE-1180 compliant, except for SSE/MMX (Skal)."

If "SSE/MMX (Skal)" is not IEEE-1180 compliant, then why it is the default option in DGindex???
How much quality do I lose when I use this algorithm???

Thanks!

[Guest]

If "SSE/MMX (Skal)" is not IEEE-1180 compliant, then why it is the default option in DGindex? Because that is what I use and I am the author; as Britney might say: "It's my prerogative". If you prefer a different one, select it and it will be saved in the INI file.

How much quality do I lose when I use this algorithm? It's completely imperceptible, as you can see by using this script:

skal=MPEG2Source("E:\tmp\manono\LloydSample.d2v",idct=6)
ieee=MPEG2Source("E:\tmp\manono\LloydSample.d2v",idct=5)
subtract(skal,ieee)

[Guest]

A few days ago I was given a 75MB 1920x1080/88 MPEG-2 .TS sample with DTS audio to try with an high-def network capable DVD player I'm currently beta testing.

While the sample provided appears to play perfectly in the affore mentioned player. When I run the sample thru' DGIndex 1.4.6 beta 2, only the video stream is de-muxed. We have two problems:

1. DGIndex does not support demuxing DTS from a transport stream. I'm not aware of any spec that defines syntax for that in the transport stream. Are you?

2. The stream sends a PMT specifying the audio as AC3. But there is never an AC3 sync word, so demuxing fails.

Unless you can point me to a spec that defines this situation, i.e., DTS audio in DVB/ATSC transport, I cannot help with this. Can you get some clarification from the guys who gave you the stream?

[Guest]

Here is version 1.4.6b3:

* Changed DGIndex/DGDecode so that when DGDecode sets the frame rate
in Avisynth, it uses the exact fractional representation for
fps_numerator/fps_denominator (changes by 'tritical').

This way, e.g., NTSC clips get the exact 24000/1001, 30000/1001, or
60000/1001 rate instead of 23.976, 29.970, or 59.94.

DGIndex was actually getting the framerate correct, but when it wrote the
d2v file, it would multiply the floating point value up by 1000 and then save
it. Then DGDecode would use that value for the numerator and 1000 for the
denominator. Now, when DGIndex gets the fps value from the lookup table it
also gets numerator/denominator values from a lookup table. Then in the d2v
file it writes, e.g.:

Frame_Rate=29970 (30000,1001)

with numerator and denominator in parentheses. DGDecode then uses those
values when setting the frame rate in Avisynth.

This change bumps the D2V file format version number to 13.

* The message box that pops up for transport streams when no data is
found has been clarified to direct the user to use the Stream menu to set
the PIDs.

* Fixed a hang when the CLI -HIDE option is used and the input file cannot
be opened.

* The video overlay is now not claimed when the -HIDE or -MINIMIZE CLI
options are used.

* File version properties were added for DGIndex, DGDecode, and DGVfapi.
If you right click on the EXE, DLL, or VFP files and select properties, you can
now obtain file version information.

* The DGIndex Information dialog now shows an overall average bitrate for
the entire previewed video in addition to the windowed value. The windowed
value now reports every 32 frames instead of every 64 frames.

http://neuron2.net/dgmpgdec/dgmpgdec146b3.zip

(I am still working on the fix for the loss of 4 bytes at the end of a demuxed video file.)

[Rockas]

(I am still working on the fix for the loss of 4 bytes at the end of a demuxed video file.)
Is this happening on every previous releases?

[Guest]

Is this happening on every previous releases? Yes. But please note that not all streams are affected, and even for ones that are, it is often harmless.

[Trahald]

usually last 4 bytes is the sequence end code. not critical for most applications.

[lexor]

hey neuron, any word on that problem of only being able to add 64 .ts files I had in beta2? I'm at work can't check out the new beta right now :(

[AllTimeSToneD]

usually last 4 bytes is the sequence end code. not critical for most applications.
Yes, thats uselly the case with Vob's but if you have a .ts as input from an atsc station its uselly the last frame which in most cases will then be a missing/undecodable B-Frame/P-Frame. :(

...problem of only being able to add 64 .ts files...
I had no problem loading 256 .ts file in beta2. Beta3 now supports up to 512 (not tested myself yet though).

Are you sure you havn't accidently loaded diffrent resolution or PID'd .ts files?

[lexor]

here is the link to my thread (which started as help call for gray line issue):
http://forum.doom9.org/showthread.php?t=103850

all .ts were consistent with each other, and I could add them all to the list, just not all at once, only in batches of 64 at a time.

/edit: ok, just got home, still can't add more than 64 at a time to the list.

[AllTimeSToneD]

Try drag and dropping them all at once in the DGindex window. I uselly just drag and drop them bcuz im to lazy to use the "File-Open" gui. :p

[lexor]

Try drag and dropping them all at once in the DGindex window. I uselly just drag and drop them bcuz im to lazy to use the "File-Open" gui. :p
holy flipping... drag&drop all at once into the main window did add them all to the list at once! but still I would expect normal add procedure to work as well.

[Rockas]

Yes. But please note that not all streams are affected, and even for ones that are, it is often harmless.
usually last 4 bytes is the sequence end code. not critical for most applications.
Thanks for the information.
Yes, thats uselly the case with Vob's but if you have a .ts as input from an atsc station its uselly the last frame which in most cases will then be a missing/undecodable B-Frame/P-Frame.
Can this, somehow, affect MPEG files too?

[Guest]

Can this, somehow, affect MPEG files too? All video demuxing is affected.

[Guest]

Beta 4 fixes a problem in setting of the frame rate in Avisynth by DGDecode (merging error when absorbing tritical's fix).

http://neuron2.net/dgmpgdec/dgmpgdec146b4.zip