DGMPGDec 1.4.0 Final!

[LigH]

* Increased the parse depth for a first pack header to 2,500,000 bytes and added additional emulation protection (fixes Moitah's weird popcorn explosion MPEG1 stream).
Might this be a reason that a member in the german board reported DGIndex taking a remarkable amount of time (> 30 s) to open an MPEG2 ES demuxed by ProjectX (so I'd guess it once was DVB)?

http://forum.gleitz.info/showthread.php?p=210026#post210026

I'll ask him to add more details here, if required.
__

@ iNFO-DVD:

:thanks: I'll post this tool in the german board for you, if you don't mind.

[iNFO-DVD]

Fine, no probs, go ahead.

[Guest]

Might this be a reason that a member in the german board reported DGIndex taking a remarkable amount of time (> 30 s) to open an MPEG2 ES demuxed by ProjectX (so I'd guess it once was DVB)?

http://forum.gleitz.info/showthread.php?p=210026#post210026

I'll ask him to add more details here, if required.
It's doubtful, but you never know. 30 sec to read 2.5Meg?

If he would like to make the stream available I'll be happy to have a look at it. I can't help directy because the link you gave is German and I'm monolingual.

[LigH]

akapuma sometimes posts here, too. I think he may use rapidshare for a "cutlet". ;)

[akapuma]

Hello,

here is the small part, announced by LigH.

The way of this part:

- recording with the technisat SkyStar2-DVB-Card and the Software DVBViewer 3.1 PRO (TS-format)
- error fixing and demuxing with ProjectX, converting to m2v (and mpa)
- cutting with MPEG2Schnitt

I need approx 30s to open this file with DGIndex 1.4.0 RC4 and RC5. RC3 and before: less then 1 second. Without cutting with MPEG2Schnitt, the problem is the same.

I hope, the attatchment is helpfully.

Best regards

akapuma

http://rapidshare.de/files/2757733/dgtest.m2v.html

[Leak]

I need approx 30s to open this file with DGIndex 1.4.0 RC4 and RC5. RC3 and before: less then 1 second. Without cutting with MPEG2Schnitt, the problem is the same.

If this already happened with RC4 it's probably not related to the parsing depth change in RC5...

Just my 0.02 €.

np: Pole - Stadt (2)

[LigH]

You are probably right... but I didn't find many changes described in detail from RC3 to RC4, except "fix for Teegedecks problem". Here, only DG will know exactly what was changed to fix this.

[Guest]

I need approx 30s to open this file with DGIndex 1.4.0 RC4 and RC5. It takes about 2 seconds on my machine with RC5.

I can't think of a possible cause right now. Is it that way with all your elementary streams? Are you loading off a hard disk?

[akapuma]

Hello,

I tested my testfile again with new installations of DGIndex:
- DGIndex 1.4.0RC2: less then 1 second
- DGIndex 1.4.0RC3: less then 1 second
- DGIndex 1.4.0RC4: approx 30s
- DGIndex 1.4.0RC5: approx 30s

I tested this whith different streams, the problem is always the same. I loaded the file directly from my harddisk. Additionaly, I tested this with different movies:

- recorded in TS, demuxed, fixed and converted in mpv with ProjectX: long time to open
- recorded in TS with another software (DVBPortal), demuxed, fixed and converted in mpv with ProjectX: long time to open
- recorded in mpg, demuxed, fixed and converted in mpv with ProjectX: long time to open
- recorded in mpg: fast time to open

I have an Athlon 1400 and Windows 98SE.

Best regards

akapuma

[Guest]

OK, people, help me out. Is anyone else getting long load times with the file that akapuma posted a link to?

[LigH]

Duron-800, 256 MB; W2KSP4:

- RC1: immediately
- RC2: immediately
- RC3: immediately
- RC4: 3 seconds
- RC5: 7 seconds

[Pasqui]

@neuron2
It takes about 3 seconds with my Athlon XP 2600+

[LigH]

Duron-800 (same system); W98SE:

- RC1: immediately
- RC2: immediately
- RC3: immediately
- RC4: 16 seconds
- RC5: 29 seconds

[sunbeam]

P4 2.8GH, W2KSP4

- RC1: immediately
- RC2: immediately
- RC3: immediately
- RC4: 5 seconds
- RC5: 10 seconds

same results with all other files handled in the same way akapuma described above.

[Krismen]

RC1: immediately
RC2: immediately
RC3: immediately
RC4: 2 seconds
RC5: 4 seconds

My CPU: AXP1700+, Os: Win XP SP2

[SeeMoreDigital]

P4 2.8GHz with 512MB RAM, WinXP Pro SP2

RC1: Immediately
RC2: Immediately
RC3: Immediately
RC4: Around 6 seconds
RC5: Around 11 seconds


Cheers

[Guest]

Thanks, people.

akapuma, do normal VOBs open fast? I.e., is it only elementary streams that are slow?

If so, I will probably restore the original pack parsing depth and add an extra open option callsed something like "deep open" that would have the longer parse depth for the files with junk at the start.

[LigH]

For me, VOBs always open immediately.

[Trahald]

* Increased the parse depth for a first pack header to 2,500,000 bytes and added additional emulation protection (fixes Moitah's weird popcorn explosion MPEG1 stream).

That helped on some files i had as well. Thank you

[akapuma]

Hello,

some tests with the RC5:

- my testfile:
approx. 30s

- my testfile, muxed with audio to a mpg (with "mplex1 with gui"):
immediately

- VOB's:
immediately

Best regards

akapuma

[iNFO-DVD]

'neuron2'

Can I just get some clarification on the 'WAV' naming convention used....

[AC3]
VTS_01_PGC_01_1 T01 3_2ch 384Kbps DELAY -31ms.ac3
[NAME]-[TRACK]-[CHANNELS]-[BITRATE]-[DELAY]

[MPA]
SVCD T01 DELAY 0ms.mpa
[NAME]-[TRACK]-[DELAY]

[WAV]
VTS_01_PGC_05_1 T01 48K 16bit 2ch.wav
[NAME]-[TRACK]-[BITRATE]-[CHANNELS]

I notice there's no 'DELAY' for wav, is that always like that?
Is that something I need not worry about?

[Guest]

The WAV files should always have 0 delay.

[Cyberia]

Can the WAV naming scheme work more like the other formats? Meaning, add a bitrate and move the channel info forward.

eg: VTS_01_PGC_05_1 T01 48K 16bit 2ch.wav becomes VTS_01_PGC_05_1 T01 2_0ch 384Kbps 48K 16bit.wav

Could you also add a frequency value for the other formats?

:D

[iNFO-DVD]

I was just messing with the MPEG-1 support and did notice something and just wondered why this happens or if it supposed to happen.

The 'Aspect_Ratio' format is usually like this: Aspect_Ratio=4:3,525 (clip was 352x288)
But just done another one and the ratio is like this: Aspect_Ratio=0.6735 (clip was 352x240)

Is it supposed to be using these 2 different formats?

Latest RC5 was used.

[Guest]

Can the WAV naming scheme work more like the other formats? Meaning, add a bitrate and move the channel info forward.
Not worth the bother. Anyway, for uncompressed audio, the bitrate and the sample rate of simple multiples of each other.

[Guest]

Is it supposed to be using these 2 different formats?
Yes, per ISO spec, as discussed earlier in the thread.

[Guest]

Here is RC6. It addresses akapuma's issue with the long open time. It now uses normal parsing depth unless the "Deep Parse" checkbox is checked on the file list dialog. Check it only to open the problematic program streams with a lot of junk before the first pack header.

http://neuron2.net/dgmpgdec/dgmpgdec140rc6.zip

[akapuma]

Here is RC6. It addresses akapuma's issue with the long open time.Thank you, it works fine.

Best regards

akapuma

[fccHandler]

VirtualDub-MPEG2 1.6.8 (http://fcchandler.home.comcast.net/stable) can open the M2M video now. It will automatically do a "deep parse" up to 3 MB if necessary (for M2M), but there should be no additional cost when opening a normal MPEG or m2v. If you're interested, the relevant code is in Mpeg2.cpp, in MPEGFileParser::Verify().

[buzzqw]

[Wrong topic]
I have several problem opening mpeg or m2v with vdubmpeg 168
but will see in other thread

BHH

[Guest]

@fccHandler

I considered something like your solution but I was worried by emulated codes in the junk content and I thought it was not worth the effort. Maybe I'll rethink that if your solution doesn't run into any problems.

Thank you for the new version of VirtualDub MPEG2. I see you have RFF handling now. Cool. Can we hope for "force film" and multiple source file support in a future version? Force film would be desirable because one doesn't have the ability to invoke external Avisynth IVTC filters.

[hartford]

Thanks!

[Guest]

Here is 1.4.0 RC7. It's a hybrid of RC4 and RC6.

* Restored the legacy support for decoding AC3 to WAV.
* Removed "Process WAV".
* Can no longer apply SRC/normalization to LPCM; it just gets demuxed.
* Implemented a scheme similar to fccHandler's to do an automatic "deep parse" for pack headers.

I'd really like to release this one. A couple guys want the source code, and I prefer to release only final code. So please beat on it. Thank you.

http://neuron2.net/dgmpgdec/dgmpgdec140rc7.zip

[scharfis_brain]

There seems to be a weird chroma problem.
Until now I saw it several times with clips that are not 100% Film.
So using Fieldoperation none.

Everytime when the pattern breaks and the vide becomes interlaced for a few frames, the chroma of the even field is swapped to the odd and vice versa.
This creates a horrible chroma-combing when I do an IVTC afterwards.

http://home.arcor.de/scharfis_brain/samples/scottz.mpg

I do not know, whether it is a problem of all of these files, so concluding the encoders are failing (I've several Files showing this very odd behaviour!)
or a problem of the MPEG-Decoders...

[Guest]

Can you please post the full script with IVTC, so I can exactly duplicate the issue?

[Guest]

I served it with Ignore Pulldown Flags and then did assumetff.separatefields. That puts DGDecode pulldown code out of consideration.

Stepping through frames after frame 274, you can see the problem in the source fields.

It's frame structure, so it's highly unlikely that the MPEG2 decoder is doing anything untoward. And the fact that it occurs only when 3:2 sections temporarily stop, even though DGDecode is ignoring pulldown, is suspicious. It just looks like strange source to me.

[ron spencer]

ac-3 to wav is back?

BTW can it do 5.1 to 2.0 WAV?

[scharfis_brain]

@neuron2:
thanks for investigating this.

It just looks like strange source to me.

hmm :rolleyes:
this isn't the only source showing this weird behaviour.
even DEFT NTSC->PAL conversions (IVTC+Speedup)
are showinc this on pattern breaks :(

one of those commonly used IVTC-machines must have a big big chroma problem...

[Guest]

ac-3 to wav is back?

BTW can it do 5.1 to 2.0 WAV? Yes. Yes (if you mean 5.1 AC3).

[mic]

@neuron2:
thanks for investigating this.

It just looks like strange source to me.

hmm :rolleyes:
this isn't the only source showing this weird behaviour.
even DEFT NTSC->PAL conversions (IVTC+Speedup)
are showinc this on pattern breaks :(

one of those commonly used IVTC-machines must have a big big chroma problem...

Some observations, for what it's worth and all...

Downloaded source file (scottz.mpg) and it seems a bit off. Looked at the orig. video in Vegas (2 versions -- 2 decoders) at various fps & frame settings. Used latest DGIndex followed by VFAPI in both Vegas & V/Dub, then AviSynth to V/Dub. Lastly tried a few methods of repair and/or conversion.

To my very untrained eye I'd guess that the downloaded video stream was partially processed (IVT) with mixed (non-consistent) flags that tended to confuse every app I tried. The frame numbers appeared consistent, but the length of the video varied by ~2 sec.. Software I used often wanted to insert frames (3:2), and these varied from black to heavily interlaced to frame doubling to std. animation tweeners.

At any rate, I was just curious with a bit of time to offer... If I'm correct in my guessing, you might be able to repair the headers &/or flags, or you could probably find a combination of settings in your normal workflow that works for you... FWIW, using the original file or ignoring flags in DGIndex, I was able to get what I thought reasonable output in V/Dub or Vegas forcing 23.976 p.

[Guest]

In my experience, the best way to flush out the last bugs is to release the final version. :)

So DGMPGDec 1.4.0 is now released. The changes from RC7 are as follows:

* Fixed a problem that caused failure of AC3 audio demuxing from a
transport stream if the audio stream was not contained in the PAT/PMT
tables.

* Fixed a problem in DGVfapi that caused conversion of AVS scripts to fail
when they delivered RGB32. [fix by 'tritical']

http://neuron2.net/dgmpgdec/dgmpgdec.html

The source code is available and a DGIndex Reference Manual by Cyberia and myself will be added shortly.

Thank you to all the contributors to this thread for your extremely valuable suggestions, bug reports, and feedback. Special thanks to tritical for his code contributions, to Cyberia for his moderating, housekeeping on the development list, keeping me honest, and reference manual contributions, and to fccHandler for his VirtualDub MPEG2, which served as a model for the dual MPEG1/MPEG2 support, and for his other helpful analyses and suggestions. Remember that aphorism about standing on the shoulders of giants? In real life, I'm a midget. ;)

I'm ready to crank 1.4.1 when the last bug reports roll in. Please don't disappoint me!

[video_magic]

Thanks a lot for your work Neuron2! :) And to all the others whose work is in this
I'm really starting to get into using it and Avisynth, I really appreciate it.

[Backwoods]

I'm ready to crank 1.4.1 when the last bug reports roll in. Please don't disappoint me!

Do you think 1.4.1 can include "Decode MPA to WAV"? sorry for asking so late but the final release now says "Decode AC3 to WAV" now and it snapped as to why the MPA wasn't being decoded from the start now. For us HDV cam users ya knaws. If not, np there is always BeSweet/Light.

[ron spencer]

hey thanks!!!! over at Videohelp they are calling this a beta in their history notes though...this is final.

thanks!!!

[Guest]

Do you think 1.4.1 can include "Decode MPA to WAV"? I'm surprised you didn't contribute to this thread:

http://forum.doom9.org/showthread.php?t=96524

I'll never say never, but it's unlikely to rise high on the priority list anytime soon.

[Backwoods]

Ah, I remember that thread when it started but I failed to keep up with it.

That thread talked me into to keep using BeSweet/Light.

[tritical]

I've got one minor bug report, line 256 in vfapidec.cpp... the fscanf format string has a %s where it should have %dx%d. It doesn't make much difference since those values are never used, but it does cause some stack corruption around the integer variables which causes an error when trying to use a debug build.

[Guest]

I've got one minor bug report, line 256 in vfapidec.cpp LOL. It's been like that since at least 1.0.12! Thanks for pointing it out.

[Doom9]

I have a feature request for the commandline if you wouldn't mind: allow the selection of multiple audio track IDs to be demuxed. Right now, it's none, one or everything. I guess one is the most commonly used, but two isn't out of the question either, and if your source has more than two, you end up wasting space for an extracted track you do not need (and if dgindex is run from a third party software, routines have to be written to re-identify the proper files (not that it is much of a problem but it would be nice to get around that from the start)).

[Guest]

@Doom9

Just out of curiosity, how does your program know which tracks to select?