DGMPGDec 1.5.3 Final

[Underground78]

Those values are read from the stream, not calculated by DGIndex. If you'd like to post a link to a stream sample, I can investigate further.

Can some streams be without this "Display Size" value or is it a bug if I never have this field filled ?

[Guest]

Please read the Users Manual that I spent so much time creating. :p

[Inventive Software]

Those values are read from the stream, not calculated by DGIndex. If you'd like to post a link to a stream sample, I can investigate further.

I will both double-check the manual and make a small sample later. ;)

[Underground78]

Please read the Users Manual that I spent so much time creating. :p

Ok, I find everything about that in the manual, sorry ... :o

[karl_lillevold]

I have a .mpg MPEG-2 HD file that seems to be pretty broken. Only Cyberlink PowerDVD can play it back. The latest DGindex crashes when opening it without saving a log file. It has been captured from broadcast HDTV, and then converted to MPEG. Unfortunately, I no longer have the original TS file.

Does anyone have any suggestions on how it can be repaired before feeding it to dgindex?

Karl.

[Guest]

Can you post a link to the MPEG so that I can see why it is failing?

[karl_lillevold]

OK. I will PM a link.

[laserfan]

...captured from broadcast HDTV, and then converted to MPEG....any suggestions on how it can be repaired...Try mpeg2repair and/or VideoReDo Plus (the latter has free 14 day full-function trial w/registration).

[karl_lillevold]

laserfan: Thanks. However, mpeg2repair appears to work only on transport streams, and VideoReDo Plus crashes :(

[Guest]

The first sequence header of the stream declares a size of 704x480 and then all the rest are 1280x720. But most software will allocate buffers based on the first sequence header. I made your stream work by cutting away the first sequence.

Is this a one-time occurrence for you?

[karl_lillevold]

Very good - thanks!

It is very understandable why it crashes then. I will cut away the beginning as well. Perhaps it was cut right at the transition between a commercial and normal content.

Yes, other streams works just fine. There should be no need to write special cases for this.

Btw, I could have probably figured it out with dgindex source code myself. I will try to complete the compilation, but I am having some trouble with the one asm file requiring ml.exe. I will edit this post once I gain access on my build system, which is out of reach right now, or edit the source code so perhaps this code is not needed.

Thanks!
Karl.

[Guest]

ml.exe is readily available. PM me if you want me to point you to it.

[karl_lillevold]

Yes, the problem is not availability of ml.exe, but which version to use. I get various errors in idctmmx.asm, and have not found an ml.exe version that works.

My latest attempt was with "Microsoft (R) Macro Assembler Version 6.14.8444".


1> Assembling: idctmmx.asm
1>idctmmx.asm(1120) : error A2008: syntax error : xmm
1> DCT_8_INV_ROW_1_s2(2): Macro Called From
1> idctmmx.asm(1120): Main Line Code


ml 8.00.50727.762:


1> Assembling: idctmmx.asm
1>idctmmx.asm(52) : error A2008: syntax error : TEXTEQU
1>idctmmx.asm(1120) : error A2070: invalid instruction operands
1> DCT_8_INV_ROW_1_s2(5): Macro Called From
1> idctmmx.asm(1120): Main Line Code


ml 6.11d:


1> Assembling: idctmmx.asm
1>idctmmx.asm(795) : error A2070: invalid instruction operands
1> movq(5): Macro Called From
1> DCT_8_INV_ROW_1(2): Macro Called From
1> idctmmx.asm(795): Main Line Code


For the older ML versions I have iammx.inc and iaxmm.inc.

I would appreciate if you could point me in the direction of the right ml version.

Thanks,
Karl.

[Guest]

Do you have VC++ 6, Visual Studio 2005 Express, or what?

[karl_lillevold]

VS 2005. The full version, not express.

[Guest]

You need version 6.15 (mine is 6.15.8803). Most people get it from the VC processor pack. You can find it on the web. Let me know if you have any problems finding it. I tested it with VS 2005 Pro and it works fine for assembling idctmmx.asm.

[karl_lillevold]

Thanks, Don! I believe I have the VC proc pack somewhere on one of my older systems, too. I just did not think to try that version.

[morphinapg]

I get "Couldn't open input file in HIDE mode. Exiting" with every file I try to open in Hide mode. I use DGindex in my ASXGui encoder. RC2 worked fine. Should I go back to RC2 or is there a way to fix this?

[Guest]

Please give me your exact CLI line. The syntax is more stringently checked, especially the = signs must all be present,especially with -IF=[...]. I'll test it in the meantime.

EDIT: Yup, it works fine and it fails as you say if I leave the = sign out for -IF=[...]

[morphinapg]

Please give me your exact CLI line. The syntax is more stringently checked, especially the = signs must all be present,especially with -IF=[...]. I'll test it in the meantime.

EDIT: Yup, it works fine and it fails as you say if I leave the = sign out for -IF=[...]

Yeah, that fixed it, thanks

[Guest]

Great! Thank you for putting the issue on the record here, as others may be unaware of the parsing change.

[tre31]

neuron2 ok, did some more testing regarding the previous colormatrix/colorimetry issues (no hints) that I had. dgindex v.1.48 does parse it correctly and the proper values are being reported (hence why I said it used too work as its an old version).

v.1.48:
Colorimetry: ITU-R BT.709

v.1.49b4
Colorimetry: ITU-R BT.709

v.1.49b14
Colorimetry: ITU-R BT.709

v.1.49b17
Colorimetry: Unknown

v.1.49:
Colorimetry: Unknown

v.1.50b2:
Colorimetry: Unknown

So hopefully that can help you narrow it down, its quite obvious that something changed between 1.49b14 - 1.49b17 that affects recognising the correct colorimetry within the mpeg2 stream as the stream does have colorimetry, and I can pretty much assume that whatever changed is affecting all dvb channels within Australia (as I haven't seen colorimetry since 1.49b14 when processing).

here's the sample (3.8mb) so you can draw your own conclusions...
http://rapidshare.com/files/102937966/CH9.1440x1080i.HDTV.AU.m2v.html

I really hope you can solve this problem. ;)
(sorry I took so long, just had other things going on...)

[tre31]

neuron2 ok, did some more testing

I hope you still have all the old versions in source otherwise it'll be somewhat troublesome, but at least I've given you as much idea as what I have on the issue this time...

[Guest]

I've fixed it. It will be in the RC4 release (not the informal performance testing one).

[opieant]

Is it possible to add an option to produce a type-1 timecode file for mkvmerge for variable frame rate input files? DGIndex is already tracking transitions between "film" and "video" modes so I imagine this can be done pretty easily and accurately.

The type-1 format generally starts with:
# timecode format v1
Assume 29.970030
The first line is a comment that happens to specify the format, is checked by mkvmerge, and is therefore required. The "Assume" line has the default frame rate.

The remaining lines in the file specify frame ranges and the actual frame rate, such as
0,571,23.976024
where frames 0 through 571 (inclusive) are at 23.976024 FPS. The next line would begin with the next frame number that has a frame rate other than the default. If a single frame has a different frame rate, that frame number is used as the start and end frame number:
2435,2435,30.0Frame rates may be integers or floating-point numbers. If a range of frames is skipped over, the default rate will be used.

Thanks :)

[Guest]

Not possible unless you do it yourself. Write a utility to generate it from the D2V file.

[opieant]

Don: I already started looking into this possibility but ran into a problem. The D2V file doesn't contain the frame rate info from the sequence headers, just one frame rate at the beginning, so this can't be done based on the D2V file alone unless I'm missing something major.

It may be possible to calculate the picture durations accurately from the flag data, in which case I can generate a type-2 mkv timecode file (specifying every picture's presentation time). I'll look into this further. At a glance it seems like one necessary flag is missing, but I haven't picked apart this type of data in ages so I'll review the standard tomorrow.

FYI, I already tried to use the "timestamps" logging feature in DGIndex to get the PTS values but I discovered that not all pictures and GOPs are showing up in that log file. I can provide a sample input if you're interested in looking into this issue.

Thanks for your help :)

[plugh]

A minor usability request:

Video menu has "HD Display" mode. (Thanks)
However when using the cropping filter, I can't switch mode.

So I have to (for example) switch to 'Top Left', invoke the cropping filter, set top and left, exit the filter, switch mode to 'Bottom Right', re-invoke the filter, set bottom and right.

It gets the job done, but is cumbersome.

:thanks:

[Zep]

I've been looking for streams like that, which allegedly mess up sync. Feel free to upload one of those.

I know I still need to get you a sample that does this but I have only capped 2 480i and no 1080i since that post. 720p basically is all I seem to cap lol. So why am I posting this? Well the one 480i had no field transitions and the other had just 2. Not enough for me to notice if sync was off or not but I did notice the frame count was different by 1 frame when I compared the 2 d2v's and why I am posting now to ask you....

Is that is normal behavior? ( The length of the fixed d2v had 1 more frame than the non fixed and the audio was 41ms shorter than video. The NON fixed one was the exact same length for video and audio )


This week I'm gonna cap some 480i stuff just for you in search of an out of sync one so I can send you a good sample.

thanks

[Zep]

Oh almost forgot. In that same 480i there appears to be a dimension change and DGindex content would change size during preview and what was there before the change stays on the very right side while the new content previews (the DGindex window stays the same size. only the content area changes. looks like it goes from 704 x 480 to 512 x 480)

I never saw that happen before. I would send it but I deleted it by mistake and it was not recoverable. This is another reason why I'm gonna find you some samples this week instead of waiting for a random cap to have problems.



Thanks

[Guest]

If you have a bad transition like:

0->0->2->2

then you can fix it by changing it to this:

0->1->2->2

So a 0 (two fields) got changed to a 1 (three fields). So two such corrections will add two fields, i.e., a frame.

It's obvious that a lot of such fixes can throw off the audio. My contention is that such errors are rare. That's why I keep asking for a stream that shows a lot of the errors. I think that it can only result from some deeper pathology (such as a broken capture card, or dropouts in a transmission due to poor signal, etc.), which is the real thing that needs fixing.

So I'm standing by that position until somebody gives me a real example. If I get one and I agree that something needs to be done, it will be possible to track the added fields and drop frames as needed to keep the AV sync.

[Zep]

If you have a bad transition like:

0->0->2->2

then you can fix it by changing it to this:

0->1->2->2

So a 0 (two fields) got changed to a 1 (three fields). So two such corrections will add two fields, i.e., a frame.

It's obvious that a lot of such fixes can throw off the audio. My contention is that such errors are rare. That's why I keep asking for a stream that shows a lot of the errors. I think that it can only result from some deeper pathology (such as a broken capture card, or dropouts in a transmission due to poor signal, etc.), which is the real thing that needs fixing.

So I'm standing by that position until somebody gives me a real example. If I get one and I agree that something needs to be done, it will be possible to track the added fields and drop frames as needed to keep the AV sync.

the last cap I saw TONS of this error was a 480i show called joes VS pros. Thousands of them and the fixed d2v made sync go way way off and by what you said above I now see why.

The non fixed one stayed in perfect sync but makes me wonder what was wrong with the video, cause when I watched it, I didn't see anything wrong. So I guess DGindex is matching the wrong fields when I stick to the non fixed one? What is the worst case video wise strangeness I should see from this?

thanks!

[Guest]

The non fixed one stayed in perfect sync but makes me wonder what was wrong with the video Me too. But if no one ever gives me such a stream, I have no hope of finding out!

[Zep]

Me too. But if no one ever gives me such a stream, I have no hope of finding out!

well I am doing some 480i tonight so fingers crossed lol


UPDATE: out of 3 hours of just rawing a 480i channel to see what it showed basically 2 corrections per hour was the average
and looked like the following give or take. (this one was from the 10 PM show)

D2V Fix Output

Field order transition: 2 -> 0
d00 5 0 547373193 0 0 0 92 a2
d00 5 0 547396729 0 0 0 90 a0 a0 a0 a0
corrected...
d00 5 0 547373193 0 0 0 92 a3
d00 5 0 547396729 0 0 0 90 a0 a0 a0 a0

Field order transition: 0 -> 2
d00 5 0 819061264 0 0 0 90 a0
d00 5 0 819076508 0 0 0 92 a2 a2 a2 a2 a2 a2 a2 a2 a2
corrected...
d00 5 0 819061264 0 0 0 90 a1
d00 5 0 819076508 0 0 0 92 a2 a2 a2 a2 a2 a2 a2 a2 a2



that was it for tonight. I will try another channel tomorrow night. Gonna be pure random chance to get one as bad
as that pro's VS joes one but I will keep at it.


thanks

[plugh]

I've got one that has me scratching my head...

transport stream file, being processed by RC2.

If I say 'save project' with 'demux all audio' and NO 'log timestamps', at the end dgindex reports bad field transition. When I let it correct it, it shows a SINGLE fix.

If I say 'save project' with 'demux all audio' and ENABLE 'log timestamps', at the end dgindex reports bad field transition. When I let it correct it, well I'm guessing, but it looks like it massages a LOT of the file. (gross metric - .d2v=1300KB, fix.txt=592KB)

Other odd things I've noticed so far:

It is reported as Film ~70%. It does this for pretty much any segment, though the number varies some. As an example, I played a short section; 'Frame type' switched back and forth between Progressive and Interlaced, Playback #=706, Coded #=592, and Field Rpts=228. (706-592=114, 114*2=228), Video Type = Film 76%

mpeg2repair generates error groups every 14 frames(?)
ie "Sequence Frame NN(12-B)" where NN is 28,42,56,...
consisting mostly of 'temporalref gap' and 'unexpected tff/rff flag change'.
At the end it reports,
0 of 273148 video frames found with errors
12 of 333624 audio frames found with errors
0 corrupted video bytes in file
18300.7 seconds of video timestamp gaps

delaycut reports 1533 errors in the dgindex created ac3 file (vs 12 above?) and oftimes has to 'rewind 170 bytes' in order to find a sync word.

Comments?

[Guest]

OK, I found the bug that causes the parsing to be different when timestamp logging is enabled.

Were there any other DGIndex issues you are pointing out to me?

[plugh]

Ah - you replaced your prior reply...
So you don't need the 500MB sample I just finished uploading?
OK, I found the bug that causes the parsing to be different when timestamp logging is enabled.Does this mean, given my description of the stream, that the 'timestamp enabled' case is the INcorrect case?
Were there any other DGIndex issues you are pointing out to me?
I'm not sure. As I indicated, when mpeg2repair parsed the stream it only found 12 bad audio frames. When dgindex extracted the audio, the resulting file was reported as having MANY more than 12 bad frames by delaycut. Further, in many of those cases, it recovered by 'rewinding 170 bytes' to find the ac3 frame sync word. I note the TS packet size is 188 bytes, and wonder if there is a relationship - like hypothetically, did dgindex extract the payload from a ts packet and put it in the ac3 file that it should not have? Or is this perhaps simply related to the parsing error you just fixed?

{Edit:} Hmmm. think I got that backwards. delaycut reports a crc error, then needs to rewind from where it expected to find the next frame; so perhaps dgindex dropped a ts packet, thus producing the crc error and causing delaycut to have to search backwards. hypothetically...{end}

AS AN ASIDE - Based upon your experience, does my description of the video characteristics sound like perhaps this has been subjected to a dgpulldown-ISH process? I note that mpeg2repair reported 273148 frames vs the DGindexed source (with 'honor pulldown flags' selected) reporting 319959.

273148/319959 ~=.85 , 25/30 ~=.83 - Just coincidence? Is that perhaps why mpeg2repair reports 'unexpected tff/rff flag change' all through the file?

Thanks in advance...

[Guest]

When you say the DGIndex demuxed audio had more errors, was that with timestamp logging enabled? Is so, then yes, it is caused by the bug I fixed.

I assume MPEG2Repair reports encoded frames, while DGIndex reports displayed frames when honoring pulldown.

If you want to give me the upload link, I'll have a look at it.

[plugh]

When you say the DGIndex demuxed audio had more errors, was that with timestamp logging enabled? Is so, then yes, it is caused by the bug I fixed.
No. I turned on timestamp in order to try to get additional info about the audio errors (have never used it before, thought it might give PTS for both audio and video frames or some such)

I just went back and tried to recreate my sequence of ops.

--I wanted both the original ac3 5.1 and a decoded stereo mix-down wave--

opened files in dgindex, quick preview.
set audio for 'decode to wave', selected channel 0
made first pass to do the 'prescale decision'
made next pass with Save Project, yeilding .wav and .d2v
made next pass with 'Demux audio only' yeilding .ac3 <=-:eek:

Note1: The Save Project gave me a field transition warning;
I said 'doit', the log showed a _single_ correction.
Note2: Neither of the above audio output files indicates the delay to be used in the filename.

Soooo... I made another short, _aborted_ "save project" pass simply to create another ac3 output file WITH the delay value in the name.

Given my uncertainty about the delay issue, My *next* intent was to evaluate the lengths of the .wav and .ac3. Delaycut will report both duration and frame count for the ac3 file. Ran it, fed it the .ac3, and it gave me Many Errors. Uh oh...

ran dgindex again, this time to demux audio via 'Save Project'
Out of curiosity / desperation, I enabled timestamps.
Did the Save, it completed, reported the field transition (again)
I said 'doit' (again), but THIS time the log showed thousands of 'fixes'

'WTF' I said...

Ran mpeg2repair on the ts files to see what IT thought.
It gave me the confusing result of lots of temporalref and rff/tff errors, but the summary at the end seemed to say the files was OK, and only had a very few audio errors.

WTWTWTF!!

To summarize:

1st save project - one field transition error
2nd save project - zillion field transition errors

demux audio, run through delaycut, mega errors

mpeg2repair, zillion messages but summary says video ok, audio has 12 errors.

:confused: At which point I posted my question

Now...
Regarding the "<=-:eek:" above...

In the process of pulling this narrative together, I discovered:
---taa daa---
"Demux Audio Only" produced my corrupt AC3 file, but
"Save Project" (without 'timestamp' enabled) produces a correct one!

The ts sample can be found here http://www.megaupload.com/?d=Z2VLHXBC

And just to make sure it doesn't get lost in the above narrative,
- neither of the audio output filenames supplied 'Delay'.

[hajj_3]

problem with a dvd of mine using 1.5.0 RC3:

D2V Fix Output

Field order transition: 0 -> 2
d00 5 0 16384 0 2 1 b0 b0 90 a0
d00 5 0 28672 0 3 1 b2 b2 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
corrected...
d00 5 0 16384 0 2 1 b0 b0 90 a1
d00 5 0 28672 0 3 1 b2 b2 92 b2 b2 a2 b2 b2 a2 b2 b2 a2

Attached is the .d2v, .bad, .ifo

if you need the original files let me know and i'll put them on my 100/100mb ftp server for you.

[Guest]

What is the problem?

[hajj_3]

if i click "yes" to fix the .d2v the audio is around 1sec out of sync, if i click no its over 1 sec out of sync. there is no delay on the .ac3 audio either: "VTS_02_1 T80 2_0ch 192Kbps DELAY 0ms.ac3"

[Guest]

If it's a fixed offset, just correct for it in the usual way. If the transition is at the beginning, try starting your project in by a few GOPs. Without a stream, It's hard to help you. People always ask me if I want the stream and I always say yes.

[RunningSkittle]

DGindex 1.5RC3 fails to open this vob cell.

It fails to load the first cell of the main movie from lethal weapon 1,2 and 3.

media player classic, and virtualdubmod open the file without issues.

http://deep.phpwebhosting.com/~mactownkrisp/archive/VTS_01_11.VOB

help!

[Guest]

It's a known problem. Well, *I* knew about it. :devil:

I'll release RC4 tonight with the fix. There was a regression in transport stream detection. RC3 thought your stream was a transport stream. Sorry for the inconvenience.

[Guest]

@hajj_3

Instead of sending me a PM with a link to a 3.5GB download, and then stopping me in the middle to say forget it, take this other one instead, try answering my questions in the thread.

Start your project in by enough GOPs to get a reasonable delay value in your filename.

[RunningSkittle]

Not an inconvenience at all! Im just glad that I wasnt over looking something simple :0

[Guest]

* Don't check for field order transitions for streams with only frame repeats.

* Use lowest numbered audio ID to expand __aud__ instead of the first audio stream encountered in the source files.

* Fix parsing with log timestamps enabled.

* Correct handling of default matrix coefficients for hints.

* Add option for beeping and focusing when save project finishes.

* Save BMP is now enabled during play/preview.

* Fix regression in transport stream detection.

http://neuron2.net/dgmpgdec/dgmpgdec.html

[morphinapg]

Works great, thanks

[Zep]

* Don't check for field order transitions for streams with only frame repeats.

* Use lowest numbered audio ID to expand __aud__ instead of the first audio stream encountered in the source files.

* Fix parsing with log timestamps enabled.

* Correct handling of default matrix coefficients for hints.

* Add option for beeping and focusing when save project finishes.

* Save BMP is now enabled during play/preview.

* Fix regression in transport stream detection.

http://neuron2.net/dgmpgdec/dgmpgdec.html



awesome! 1 fix and 1 option in this release rock for me. this is the release I have so been waiting for!


thanks!