DGMPGDec 1.5.8

[neuron2]

Full source code release with only a minor fix for a DGIndex crashing problem with some pathological streams.

http://neuron2.net/dgmpgdec/dgmpgdec.html

[Groucho2004]

Hi Donald, thanks for this release!

I did a little experiment creating a VC6 project from the source code for DGDecode you provided and compiling dgdecode.dll.

I created a simple d2v and avs from a mpeg2 clip and ran the script through a simple AVISynth benchmark tool I wrote some time ago.

With the DLL you released I get about 125 fps, with the VC6 DLL I compiled I get about 190 fps! These tests were run on a Core 2 E7400 @3.4 GHz.

I also tried VC7.1 with all kinds of switch combinations but the VC6 compile is simply much faster.

You can try it yourself:
My benchmark tool: AVSInfo (http://www.iol.ie/~schubert/AVSInfo.zip)
The VC6 (SP6) DLL: DGDecode (http://www.iol.ie/~schubert/DGDecode.zip)

I guess when one uses heavy filtering this won't matter that much but for a simple script it makes quite a difference.

[neuron2]

My DGDecode.dll is built with VC6. Maybe you are running into differences due to caching.

[Groucho2004]

My DGDecode.dll is built with VC6. Maybe you are running into differences due to caching.

What kind of caching? Have you tried the DLL I uploaded?

Just for reference, here are the tools I use:
VC6, SP6 with processor pack
MASM 6.15.8803
NASM 2.07

[Groucho2004]

I figured out where the difference comes from. I changed the processor type in "Code generation" from "Blend" to "Pentium Pro". :)

[neuron2]

What processor do you have?

[Groucho2004]

Core 2, E7400 @3.4 GHz. It may not make such a difference on an AMD chip.

[lansing]

found a bug, when I'm previewing the video with either F5 or F6, when I scroll up in my mouse scroll, the information window disappear while the preview process was still running

[neuron2]

Fixed for next release. Thanks for the report.

[~Revolution~]

This might not be the place to ask this but I don't quite understand why DGIndex reports some PAL sources as Interlaced (Field/Frame) when I can clearly see individual, progressive frames after selecting the "Ignore Pulldown Flags" option.

[neuron2]

Because as the user manual states (and as has been discussed here many times) that field reports how the stream was encoded, not what the actual content is.

[~Revolution~]

I just read a lot of material of soft vs hard telecine and mpeg-2 flags and such. I'm sorry for the stupid question. It all seems to make sense now. I have another question however. I have been searching all over and cannot find a precise and clear way of how mpeg-2 flags exactly work. So basically in a soft telecine video the actual content is 23.976 progressive (slowed down from 24) and then with flags this can be converted on-the-fly by the decoder to show 29.976 fps or 59.94 fields/s? So the actual video on the dvd is not divided up into fields but are "artificially" created by the instructions from the flags? Also if this is true then why do dvd studios still use hard telecining vs soft telecining for material originating from 24 fps film?

[neuron2]

I have been searching all over and cannot find a precise and clear way of how mpeg-2 flags exactly work. See here:

http://neuron2.net/library/mpeg2/iso13818-2.pdf

Also if this is true then why do dvd studios still use hard telecining vs soft telecining for material originating from 24 fps film? Because they receive material already hard-pulled-down, or they are incompetents, or both.

Please start new threads in the appropriate forums for these general questions.

[~Revolution~]

:thanks: You're awesome :)

[Wilbert]

Full source code release with only a minor fix for a DGIndex crashing problem with some pathological streams.

Thanks neuron2!

[BuHHunyx]

---------------------------
DGIndex.exe - Entry Point Not Found
---------------------------
The procedure entry point AttachConsole could not be located in the dynamic link library KERNEL32.dll.
---------------------------
OK
---------------------------

[Groucho2004]

---------------------------
DGIndex.exe - Entry Point Not Found
---------------------------
The procedure entry point AttachConsole could not be located in the dynamic link library KERNEL32.dll.
---------------------------
OK
---------------------------

You could try the DGIndex I built with VC6, here (http://www.iol.ie/~schubert/DGIndex.zip).

[jpsdr]

Just for reference, here are the tools I use:
VC6, SP6 with processor pack
MASM 6.15.8803
NASM 2.07
With VC6, i was staying at the time at SP5, beacause SP6 doesn't have the processor pack, only the SP5.
Do you know where i can get SP6 + processor pack for SP6 ?
Thanks.

[Groucho2004]

With VC6, i was staying at the time at SP5, beacause SP6 doesn't have the processor pack, only the SP5.
Do you know where i can get SP6 + processor pack for SP6 ?
Thanks.

Here (http://support.microsoft.com/kb/872907) is one solution.

You can also unpack the processor pack files, copy them to the appropriate folders in your VC6 installation and then import the reg file.

[jpsdr]

Ok, thanks, i'll try just by curiousity...
Thanks.

[osgZach]

Hi Don,

Is there any chance of an x64 Dgdecode.dll for use with JoshyD's x64 port?

We have one version but it's a very old 1.4.6 I believe..

[neuron2]

You have the source code. Can't you just compile it for x64?

I don't have a 64-bit OS nor do I wish to install one at this time.

[Stephen R. Savage]

Well, that depends on if all the assembly is compatible with Win64. However, it can't hurt to try.

[neuron2]

Good point. Thank you.

[Terka]

hi neuron2,
has DGMPGDec some option to automatically split the vobs to episodes? (more d2v as output)
(This children dvd has many 10minutes 'episodes'.)

[osgZach]

Use Dvd Decrypter or another utility that lets you rip in IFO mode.

You can select individual PGC's, and even if they are all stupidly under the same PGC, once you figure out how many chapters each episode has, its easy enough to rip them into their own VOB's one at a time.

Or you can use the [ and ] brackets in DGindex to select a Range.

[Terka]

sorry,for asking, have no experience with ripping.
i already have vobs done, do i need to rip again? or is it possible to split automatically?
what is PGC?
how many chapters each episode has- must it be always the same (on 1dvd)?

[Inspector.Gadget]

Terka, if you ripped in FILE mode, use PGCDemux against each IFO file and each title consecutively to get either demuxed streams or new VOBs that pertain only to each title (episode). This is surprisingly straightforward; learning every function of PGCDemux takes about 5 minutes of trial and error. You can then feed each new VOB set or demuxed M2V to DGIndex and wind up with an index file and audio/subpicture files relevant to each episode. Since this is getting further away from DGIndex, please start a new thread in Newbies or Decrypting and we'll all be happy to answer your non-DGIndex questions there.

[Terka]

ok thank you!
http://forum.doom9.org/showthread.php?p=1382927#post1382927

[BuHHunyx]

You could try the DGIndex I built with VC6, here (http://www.iol.ie/~schubert/DGIndex.zip).

Same error - because function AttachConsole missing from kernel32.dll in w2k.

[Groucho2004]

Same error - because function AttachConsole missing from kernel32.dll in w2k.

It seems that the problem is that the code for "AttachConsole()" is still there although it's not being used at runtime on W2K.

Please re-download DGIndex from my previous post, I have commented out the code that causes the problem.

Let me know if this works.

[neuron2]

I have made a test version that should now operate correctly under Win2000 (AttachConsole() is linked dynamically only when supported, rather than statically as before). You won't have the progress reporting under Win2000 but it should not crash. Please advise of your test results under Win2000, XP, Vista, and Win7. Thank you.

http://neuron2.net/misc/DGIndex158b.zip

Groucho2004's fix is not acceptable because it removes the progress reporting on the OS's that do support AttachConsole().

[Groucho2004]

Groucho2004's fix is not acceptable because it removes the progress reporting on the OS's that do support AttachConsole().

I know it's not a fix, this was just supposed to be a test to isolate the problem.

[neuron2]

I know it's not a fix, this was just supposed to be a test to isolate the problem. I understand, but there never was any doubt about the nature of the problem.

As long as I am actively developing and fixing things, I prefer not to have third-party builds made as it will cause confusion. If you insist on making builds, I request that you not release executables with the same version numbers I use. In fact, I prefer that you rename the executables as well.

Thanks for your understanding and I do understand you are trying to be helpful, which I appreciate.

[BuHHunyx]

I have made a test version that should now operate correctly under Win2000 (AttachConsole() is linked dynamically only when supported, rather than statically as before). You won't have the progress reporting under Win2000 but it should not crash. Please advise of your test results under Win2000, XP, Vista, and Win7. Thank you.

http://neuron2.net/misc/DGIndex158b.zip

Groucho2004's fix is not acceptable because it removes the progress reporting on the OS's that do support AttachConsole().

Great - now it works! Thanks!

[neuron2]

Excellent to hear. Thank you for the feedback. I'll go ahead and release this as well as port the progress reporting to the DGNV tools.

[turbojet]

Could you add 'touch docking' the info window like dgnv has?

[neuron2]

Sure. Thanks for the suggestion.

[turbojet]

Your welcome and thanks for adding it.

[JEEB]

Hello there.

While helping a friend of mine compress his television captures, I found out that dgindex's audio demuxing doesn't really cope fine with certain types of ISDB-S (BS satellite channel) streams where the amount of audio channels gets switched on-the-fly. (short sample (http://x264.fushizen.eu/files/switchingaudio.ts))

Is this something that can't possibly be supported with demuxing, or is it workable with? Currently the workaround would be to cut the transport stream where the 5.1 part starts/ends (there's a specific tool for that as well) and have that read by dgindex.

[~Revolution~]

^Wow, this is a first I have seen. I was actually wondering how to do that in an mkv lol

[neuron2]

^Wow, this is a first I have seen. I was actually wondering how to do that in an mkv lol What are you referring to? Do *what* in an MKV?

[JEEB]

What are you referring to? Do *what* in an MKV?

He probably means switching the amount of audio channels within a file on-the-fly, just like in the transport stream sample I posted here (http://forum.doom9.org/showpost.php?p=1384315&postcount=40) -- the last post on the previous page if you use the default amount of posts per page.

And yes, something actually plays that file correctly, if you have any doubts >_> (MPC-HC's internal decoder)

[~Revolution~]

Because as the user manual states (and as has been discussed here many times) that field reports how the stream was encoded, not what the actual content is.

I was wondering as a follow-up to your response, when you say it shows how the stream was "encoded" but may not be what the "actual" content is...does that mean that the MPEG-2 Encoder sort of "tagged" the material incorrectly or something? I don't quite understand how the encoder can say it encoded interlaced but the actual stream is progressive unless the encoder tagged it wrong or something. By the way I must say that your DGPulldown and DGIndex softwares are simply God send. :)

[neuron2]

"The actual content" refers to whether there are unique pictures at the field rate (interlaced) or at the frame rate (progressive). This depends purely on how the video was sampled when created.

When I talk about how it is encoded, it is a technical issue of how chroma is sampled, and how fields are represented in the stream. You can choose an interlaced way of encoding but feed it with progressive video as defined in the first paragraph. You can also choose a progressive way of encoding but feed it interlaced content (as defined above). Or you can do it correctly and match them up.

So the two concepts (content versus encoding) are not the same.

[~Revolution~]

I understand now :thanks:

[Zep]

how do i turn off auto dv2fix when using the CLI?


thanks :)

[neuron2]

Not currently possible.

[Zep]

Not currently possible.


darn. ok I'll figure out a workaroud since i have an automated setup on multi segments of which i have no control over so what happens is the auto dv2 fix makes some segments produce tons of glitches (i.e. green blocks and strange stuttering) that the non fixed d2v does not cause.


anyway thanks for the great app!

[neuron2]

If you are using a batch file, then just delete the "fixed" one and rename the "bad" one.

[LigH]

Recently I read a statement that some people recommend to disable the internal deblocking filter of DGMPGDec, even though it has the advantage of knowing the quantization factors per block ... but it has the disadvantage of doing a rather simple job. Other deblocking algorithms are probably smarter, but lack of knowledge about the original quantization when they are used after MPEG2Source(CPU=0).

I wonder if you could imagine any way to support other deblocking filters somehow -- maybe implement them yourself; maybe implement a kind of plugin system for postprocessing functions; maybe find a way to submit hints along the clip in AviSynth; ...

Just an idea. Just curious if your reply is anything else than "not worth the efforts".

[neuron2]

"other deblocking filters"

As long as your request remains this vague, there is no way for me to respond. Specifically, what deblocking algorithm(s) would you like to see implemented?

[LigH]

I have some issues finding the thread again - since our german board was updated to version 4.0.3, the search engine is corrupted, the index incomplete... But I hope to be able to pull Didée here so he can tell you his opinions.

If I am not completely wrong, comparing with ffdshow's deblockers, DGMPGDec uses Nic's deblocking routines (with the h/v thresholds), which are quite fast but rather simple. Alternatives would be:

- mplayer deblocking
- mplayer precise deblocking
- fast SPP deblocking
- precise SPP deblocking
- precise SPP deblocking with soft threshold

I don't know much about them, except that SPP is quite slow but probably rather smart in comparison. But I could not link any proof for it. I hope Didée can tell you more...

[neuron2]

OK, waiting for Didée...

Not Godot.

[Didée]

Sorry, but I don't really want to make any "please-implement-THIS" recommendations. My point of view simply is that Nic's deblocking routines are literally past century's technology, and just aren't state-of-the-art anymore ... myths are hard to kill, and many still do believe that mpeg2source's deblocking would be preferable because it's "internal" and because it "knows" the quantizers. Sure, it knows ... just that it usually smooooths-away too much detail nonetheless.

The most painless implementation, probably, would be to just use the Deblock() code that is already contained in DGDecode. That would "only" need some halfway-reasonable mapping for the quants (1-31 <> 1-51), and of course, adaption for the case of interlacing.

It's not the option that I personally would like the most, but of all available options it's probably the one that could be done most easily.


For the record, a small comparison of source vs. mpeg2source(CPU=4) vs. "blind" Deblock() vs. "blind" deblock_qed():

ClickMe (http://www.mediafire.com/file/tnm3cmnim5m/deblocking.avi) (1.4MB, MediaFire) (Use single frame stepping to compare.)

[neuron2]

It's not the option that I personally would like the most I'll bite. What *is* the option you would like the most?

[Didée]

Could be it's not the best time to ask that ... all past years I still have been on a CRT monitor, and only now I'm about to switch to a flat panel, finally. now, CRTs are forgiving and inherently blurry, whereas TFTs are without mercy and inherently (too) sharp. Hence, my point of view might change in the future. ;)

But seriously ... to me it seems that the AVC-deblocker overall is a reasonably good basis. For the case of slight up to medium blocking, I do like the idea as it's implemented in the deblock_qed function (do the major deblocking only on block boundaries, but only little in the block interior areas - this still removes most of the visible blocking, but keeps much more of observed detail). However, when there is some "more severe" blocking, this strategy turns out not very good, and a plain Deblock(quant=big_enough) seems more pleasing.

So, while still using the basis of the existing AVC-Deblock() routines, a fairly sophisticated enhancement could go like this:

1) the deblocker uses Deblock() with adaptive quantizers acc. to the Q's of the Mpeg2 frame/block quantizers

2) the deblocker works at different strengths on block borders vs. block interiors
(actually, my qed() function uses DCTFilter to perform an 8x8-block-blur on the difference that the initial deblock() did achieve, and uses that blurred difference instead of the original difference for the inner 6x6 of a block. A lowpass, that is. Maybe DCT isn't the best option, but it was the easiest and fastest that I could only do at the script level.)

2a) this difference is modulated acc. to the actual block quantizer: for blocks with low quant, deblock mostly the borders. With increasing block quants, start doing more filtering to the interior regions, up to "full work" at some certain high quant and above. It would seem feasable to allow to specify a lower and upper threshold, to allow more control about how careful or aggressive the deblocking acts for any given block quantizer.

Huh, why do things aways sound so complicated once you try to put them in words ...

The versatility would be that, together with the quantizer thresholds, it would be possible to modulate the "smartness" from "too weak because tooo careful" at the one end, up to "same as good old naked deblock()" at the other end. No reason for anyone to complain, because everyone could set it as weak or as forceful as wished.

[LigH]

Yikes. One and a half year gone without considering this idea again.

So I created a thread in Donald Graft's forum (http://neuron2.net/board/viewtopic.php?f=7&t=190) to remind us on it, and have it done, eventually ... hopefully ...