DGMPGDec 1.4.0 Final!

[Guest]

Here is the next beta. It now can get MPEG audio out of an MPEG1 program stream. I addressed the window problem with tiny MPEG1 files by implementing a minimum window display size for DGIndex of 320x240. I was able to make it size correctly for (e.g.) 160x120, but the trackbar becomes a joke. So this seems like a reasonable solution to always have a usable trackbar.

Note that the chroma siting issue affects only conversion to RGB so it affects only the built-in conversion filter YV12toRGB24 and VFAPI. I would like to remove the built-in conversion filters (YV12toYUY2 and YV12toRGB24) because Avisynth has better ones. Any objections? I will modify VFAPI in a next beta.

Still, there is an interesting issue raised. If you can decode MPEG1 or MPEG2 and send it to Avisynth and then somewhere later a conversion to RGB is done, how does that conversion know whether to use MPEG1 or MPEG2 chroma siting??? Fortunately, the difference is very small and perhaps not of practical significance.

http://neuron2.net/dgmpgdec/dgmpgdec140b2.zip

Your feedback on this beta will be greatly appreciated.

[SeeMoreDigital]

DGIndex does not modify the GOP in any way. Can you please follow this up with mpucoder? I did and his reply was: -

Mpeg-1 has always been possible. But the same GOP constraints apply. This is not a Muxman restriction, but DVD. And considering that mpeg-1 has only frame pictures, more than 36 pictures means morethan 36 frames - twice the limit for DVD.
Make sure gop headers are not being removed - look at the original DVD with VobEdit to see how often [gop] occurs in the left pane. It should happen every I frame.And his more recent reply was: -
A gop must start with an I frame, so if there are not enough I frames to make shorter gop's then the video must be re-encoded.
There is a reason for the gop length restriction, and it has nothing to do with picture quality (arithmetic error accummulation) or any decoder related issues. The gop length restriction sets the maximum size of the most basic unit used to compose a VOBU. And VOBU's in turn have length restrictions as well of 0.4 to 1.0 second except the last of a cell, which may be as long as 1.2 second. Having gop's longer than 0.6 seconds (the difference between the shortest and longest allowable VOBU) would create situations where it is impossible to compose a compliant VOBU.
And a gop longer than 36 frames is either 1.2 or 1.44 seconds long, depending on television standard, which is longer than a legitimate VOBU.
If DVB streams are not DVD compliant then they can't simply be stuffed into a vob to be played on a DVD player, they must be processed before authoring.
Cheers

[Guest]

Thank you for that useful information, SeeMoreDigital! It does not, however, appear relevant.

You said that you demuxed the video from a DVD. It must therefore have DVD-compliant GOPs. Can you please make that point to mpucoder?

Can you please upload a chunk of the VOB to my server so that I can analyse it? Thank you.

[len0x]

Aspect ratio doesn't seem to be right for mpeg1 sources. For instance for 352x288 PAL source DGIndex displayed 4:3,625 when its clearly should have been 4:3...

[len0x]

More about AR in mpeg1 sources: I've got a few very weird ones - AR is displayed as 0.6735 (30fps 320x240) which is strange coz it doesn't have usual x:y notation at all. Its probably wrongly mastered (its still should have been 4:3) but it would be nice to see format of AR line in d2v file in generic form. Also what is the decimal separator used for AR values? It seems that sometimes its comma (3,625) and sometimes its dot (0.6735)...

[SeeMoreDigital]

Hi len0x,

I have a couple of PAL sources, that are both described as being 4:3, 625 even though they have been professionally encoded using different resolutions: -

http://img99.echo.cx/img99/1148/pal43352x2886dj.png

http://img39.echo.cx/img39/8944/pal43368x2243oi.png


And my NTSC sources are described as being 4:3, 525: -

http://img99.echo.cx/img99/5012/ntsc43352x2400gl.png

It seems as though the 625 and 525 expressions are with regard to the analogue vertical line count.


Cheers

[len0x]

ah, that explains it! I think that vertical line count could be removed from AR line as its not really needed and just messes up parsers :) Still wondering about AR of 0.6735 though...

[SeeMoreDigital]

ah, that explains it! I think that vertical line count could be removed from AR line as its not really needed and just messes up parsers :) Still wondering about AR of 0.6735 though...Which reminds me... Does anybody know of any tools that can re-write the PAR/DAR codes for Mpeg1 streams - without re-encoding?

I've tried ReStream, which although the developer says only works with Mpeg2 streams, it does seem to be able to change the PAR/DAR codes of Mpeg1 streams to either 1:1 and 16:9... but not 4:3.


Cheers

[midnightsun]

it looks to me as if 4:3,625 means 4:3 PAL (625 being the scan lines of a 50Hz PAL tv) and 4:3,525 means 4:3 NTSC (525 being the NTSC counterpart of 625 as explained above)

[Guest]

The Aspect Ratio field just reports the value from the ISO spec as I mentioned in an earlier posting. You can look at the table in the spec for details. Yes, ",625" means 625 lines, etc. That is the way it is described in the spec. There are oddball values without X:Y counterparts. That's why you see the funny decimal. Why it was used by the encoder I cannot tell you.

[Guest]

Beta 3 adds multiple instantiation for DGVfapi. Also, DGVfapi can now convert AVS scripts in addition to D2V files! Sort of a bundled AVS2AVI, if you will. Please give a hearty round of applause to tritical for contributing these enhancements. :thanks:

http://neuron2.net/dgmpgdec/dgmpgdec140b3.zip

This is close to my release feature set. I want to integrate DGParse and DGFix and add GOP structure printout in DGParse output. Is there anything else that should be considered before drawing the line on the 1.4.0 release?

[mimage]

Beta 3 adds multiple instantiation for DGVfapi. Also, DGVfapi can now convert AVS scripts in addition to D2V files! Sort of a bundled AVS2AVI, if you will. Please give a hearty round of applause to tritical for contributing these enhancements. :thanks:

http://neuron2.net/dgmpgdec/dgmpgdec140b3.zip

This is close to my release feature set. I want to integrate DGParse and DGFix and add GOP structure printout in DGParse output. Is there anything else that should be considered before drawing the line on the 1.4.0 release?It works so great for importing AVS files into Vegas!! Thanks a bunch for this release, it has made my day! :)

[kikounet]

More about AR in mpeg1 sources: I've got a few very weird ones - AR is displayed as 0.6735 (30fps 320x240) which is strange coz it doesn't have usual x:y notation at all. Its probably wrongly mastered (its still should have been 4:3) but it would be nice to see format of AR line in d2v file in generic form. Also what is the decimal separator used for AR values? It seems that sometimes its comma (3,625) and sometimes its dot (0.6735)...
VideoCD standard never had complied to 1:1 Pixel Aspect Ratio (PAR)!
4/3 is 1.3333 but 352/288 is 1.2222 (PAL) and 352/240 is 1.4666 (NTSC)
Professionaly encoded VideoCD complies to standards, so we have to deal with that, sorry... ;)

[len0x]

The Aspect Ratio field just reports the value from the ISO spec as I mentioned in an earlier posting. You can look at the table in the spec for details.

Can you point me to link with the spec, please? (is it hosted in your library?)

*Edit* I found the document on your website (mpgvideo.doc), but the table for pel_aspect_ratio seems to have mostly decimal values without corresponding AR like 4:3 for instance. How do you get values like 4:3,525 from it?

[guada 2]

Neuron and Tritical

Very Big and Good work. :thanks:

Good bye ;)

[Guest]

I found the document on your website (mpgvideo.doc), but the table for pel_aspect_ratio seems to have mostly decimal values without corresponding AR like 4:3 for instance. How do you get values like 4:3,525 from it? Here's the table from the document:
pel_aspect_ratio height/width example
0000 forbidden
0001 1.0000 VGA etc.
0010 0.6735
0011 0.7031 16:9, 625line
0100 0.7615
0101 0.8055
0110 0.8437 16:9, 525line
0111 0.8935
1000 0.9375 CCIR601, 625line
1001 0.9815
1010 1.0255
1011 1.0695
1100 1.1250 CCIR601, 525line
1101 1.1575
1110 1.2015
1111 reserved I have a later version of the spec at works that replaces the CCIR601 with 4:3.

[scharfis_brain]

are you sure, that those values are really the correct ones?

they all seem to be generic PAR, which is not recommended by ITU...

[SeeMoreDigital]

The 4:3 and 16:9 (NTSC/PAL) values are exactly the same settings I use when adding DAR signalling to my Mpeg4 streams.... nice!

[len0x]

Here's the table from the document:
I have a later version of the spec at works that replaces the CCIR601 with 4:3.

Ah, that explains it. Thanks.

[Guest]

are you sure, that those values are really the correct ones?

they all seem to be generic PAR, which is not recommended by ITU... I just display the contents of the field as per the MPEG1 spec. Are you asking if the spec is correct?

[scharfis_brain]

the problem I saw is just this:

PAL-VCD has 352x288 pixels, that need to be streched te 384x288

the table above shows:
1000 0.9375 CCIR601, 625line

but 352 / 0.9375 = 375,7 ???

the same for NTSC-VCD:
352x240 is squeezed to 320x240

the table shows:
1100 1.1250 CCIR601, 525line

352 / 1.125 = 312,9

So I am asking:
what is going on here?

[Arachnotron]

PAL-VCD has 352x288 pixels, that need to be streched te 384x288

the table above shows:
1000 0.9375 CCIR601, 625line

but 352 / 0.9375 = 375,7 ???352x288 PAL VCD has a playback sample rate of 6.75 MHz, so the line length is 352/6.75 = 52.14815 µs. Square pixel 384 x 288 describes only 52 µs. So you need to crop of 2 pixels first before stretching.

[Arachnotron]

@Sharfi

This (http://groups.google.com/groups?selm=9704042219.AA21866%40python.pictel.com) link seems to explain where your 375 came from. I found it amongst the references at the end of this (http://www.uwasa.fi/~f76998/video/conversion/#4.11)page.

[edit] it seems that somewhere along the way, they *defined* 352 pixels = 384 sq. pixels, even though with the VCD sample of 6.75 rate, 351 pels equals 384 as a sort of compromise to *simplify* things. (note that in this post the PAL acive area is set to 704 13.5 MHz pixels, while that should be 702)

So now there are two sets of values: those based on sample rates and the BT471 4:3 active window of 52 µs x 576 lines and those based on the assumption that the PAL active window is 704x576/352x288

I imagine something similar happend with NTSC.

[edit 2] 0.9375 is based on the (mistaken) assumption that the 720x576 DVD frame maps to a 4:3 area, so equals 768x576. That is of course wrong, it should either be 702x576 (if you want to be exact) or 704x576 (if you follow the compromise above)

[Cyberia]

FYI - Here are two links to some very good Aspect Ratio conversion pages:

http://www.mir.com/DMG/aspect.html

http://www.uwasa.fi/~f76998/video/conversion/

[Guest]

tritical sent a bug fix for the VFAPI support. I also added something. See if you can find it. :)

http://neuron2.net/dgmpgdec/dgmpgdec140b4.zip

[SeeMoreDigital]

Cool....

http://img185.echo.cx/img185/2347/dgindex140b44wo.png

....what do I win?


Cheers

[mp3dom]

I have a problem with the 1.4.x version of DGIndex. I have a 50Mbps MPEG-2 (all I frames) produced with Pinnacle Chrome HD. When I open the 50 Mbps file in DGIndex I have a bad preview (all colors seems to be wrong) and the creation of D2V require a LOT of time (about 5 hours for a 30 min. movie).
This problem doesn't appear with the 1.3.x version (the D2V creation require about 5 min).

[Guest]

I have a problem with the 1.4.x version of DGIndex. I have a 50Mbps MPEG-2 (all I frames) produced with Pinnacle Chrome HD. When I open the 50 Mbps file in DGIndex I have a bad preview (all colors seems to be wrong) and the creation of D2V require a LOT of time (about 5 hours for a 30 min. movie).
This problem doesn't appear with the 1.3.x version (the D2V creation require about 5 min). Please upload the file or a portion of the beginning to my FTP server and notify here when it is there. Up to a couple hundred MBytes is fine.

Also, can you fill in the x and tell me exactly which was the first version to exhibit the problem?

Also, please see if it is being incorrectly decoded as MPEG1 by looking at the Profile field of the info dialog during preview. Does it say "[MPEG1]"?

[E-Male]

is there a way to add many vob files at once via command line (i mean 100+)?

thx

[Cyberia]

is there a way to add many vob files at once via command line (i mean 100+)?

thx

RTM. Just a little poking around and you would have solved this for yourself. Read the file "CommandLine.txt" in the directory where you installed DGMPGDec.

Either the -BF or -AIF commands should be what you're looking for.

Admittedly, the information pertaining to DGIndex is scattered around several files/web pages/secret caches/etc. I am working on an actual DGIndex manual that will centralize all this witchcraft into one location.

But still, is "CommandLine.txt" not obvious enough?

[E-Male]

in my commandline.txt there only are 2 input options:
-AIF <-- won't work, the filenames aren't 'numbered' enough
-IF <-- won't work either, or can i put 100+ filenames in one command-line

EDIT:
my file was outdated (and the new updates don't contain commandline.txt)
found -BF

[Guest]

my file was outdated (and the new updates don't contain commandline.txt) The beta releases don't include all the documents. Go back to the last released version; in this case, 1.3.0. Sorry for the inconvenience.

[Guest]

I have a problem with the 1.4.x version of DGIndex. Are you still there? I'm getting a release package together and I'm really keen to get this issue resolved before the next release. I responded to your post with a query for some further information (see several posts above). Thank you.

[SCIF]

neuron2, update link for DGMPEGDec 1.4.0 beta 4 at your home page.
Thanks for your great work.

[Guest]

neuron2, update link for DGMPEGDec 1.4.0 beta 4 at your home page.
Thanks for your great work. Thanks for the pointer, but I don't update the home page until I'm reasonably confident of the stability. You Doom9'ers are the guinea pigs. Right now, I'm waiting for mp3dom to elaborate on his problem, because if it's true, it is a big deal.

[E-Male]

The beta releases don't include all the documents. Go back to the last released version; in this case, 1.3.0. Sorry for the inconvenience.
i now have 1.3.0 and beta 4 copied over it
thx

[Guest]

I've now reached feature cutoff for 1.4.0 with this beta. This version adds the following changes:

* The integrated DGParse info now shows the GOP structure.

* Fixed a bug such that during preview the info dialog did not show the audio track information for transport streams and PVA streams.

* The Detect PIDs dialog now has a "Done" button.

http://neuron2.net/dgmpgdec/dgmpgdec140b5.zip

Now we are in bug fixing mode only for 1.4.0. So pound away!

[Guest]

This beta fixes mp3dom's reported bug.

http://neuron2.net/dgmpgdec/dgmpgdec140b6.zip

Everybody should get this correction because b5 is significantly broken. I've also corrected the D2V format document after Cyberia pointed out several problems in it.

Thanks mp3dom for catching this bug, and Cyberia for the corrections.
:thanks:

[Guest]

This one fixes some bugs in the integrated Parse D2V feature, and enhances it to properly show the effect of the field operations.

http://neuron2.net/dgmpgdec/dgmpgdec140b7.zip

[len0x]

Does current version open DAT files (VCD) directly without renaming to mpg extension?
(I'd try myself but don't have any VCDs at hand).

[Guest]

Does current version open DAT files (VCD) directly without renaming to mpg extension? Yes, but they won't show up in the open dialog unless you choose "All files" in the "Files of type" pulldown, or drag them onto the DGIndex window. CLI of course can specify them directly.

BTW, I've seen an MPEG1 that had a height not a multiple of 16. But DGMPGDec is riddled through with the assumption that the display width/height are the same as the encoded width/height. The result is a black bar at the bottom. The workaround (for now) is to put Crop() in the script.

[len0x]

ok, thanks.

[Cyberia]

FYI: drag-n-drop also works for dat files.

[Guest]

This addresses a bug in transport parsing. For details, see here: http://forum.doom9.org/showthread.php?t=93190

DGMPGDec 1.4.0b8:

http://neuron2.net/dgmpgdec/dgmpgdec140b8.zip

[Guest]

Here is 1.4.0 release candidate 1. The changes from beta 8 are as follows:

* Removed YV12toYUY2() and YV12toRGB24() because they duplicate Avisynth native filters.

* Changed the LumaFilter() offset parameter default to 0 (from -2).

http://neuron2.net/dgmpgdec/dgmpgdec140rc1.zip

[Cyberia]

I thought we had those colorspace converters because AviSynth did not have the necessary information to upsample interlaced formats correctly some of the time? (ie: PROGRESSIVE_FRAME flag)

EDIT - See this thread: http://forum.doom9.org/showthread.php?t=76609&highlight=YV12toYUY2%28%29

EDIT2: I remembered in the old MPEG2Dec3 manual the lumaoff parameter was set to -2 with a comment "for iago", so I did a search for "iago mpeg2dec3" and found this thread: http://forum.doom9.org/showthread.php?t=31301&page=7&pp=20&highlight=iago+mpeg2dec3 which basically says MarcFD set the lumaoff to -2 to avoid luma blocks. It is a long thread, but just look for iago and MarkFD's comments.

Summary: iago did some extensive testing and saw blocks in the black areas, but seting lumaoff=-2 tweaked the luma just enough to remove the blocks without changing the original image in a perceptable way. MarkFD accepted it, but thought it was not the best solution.

[Guest]

I thought we had those colorspace converters because AviSynth did not have the necessary information to upsample interlaced formats correctly some of the time? (ie: PROGRESSIVE_FRAME flag) Nope. Those filters did not, and could not, use the progressive_frame flag. They receive decoded video from MPEG2Source(). And anyway, if you need that capability, you use upConv=true.

Summary: iago did some extensive testing and saw blocks in the black areas, but seting lumaoff=-2 tweaked the luma just enough to remove the blocks without changing the original image in a perceptable way. MarkFD accepted it, but thought it was not the best solution. Users can set -2 if they like it. Mention it in the manual if you like. But it sounds silly to me.

[ARDA]

@neuron2

I address you about this subject for second time
First time here http://forum.doom9.org/showthread.php?p=655698#post655698
As I think I couldn't make me understand I''ll try to be more clear now

LumaFilter had serveral bugs and special features

First time a bug was detected was about LumOffsetMask and LumGainMask
http://forum.doom9.org/showthread.php?p=322136#post322136
I've spotted to Nic but for what I've seen just half of the bug has been
fixed.The way the variables were declared (__declspec(align(8)) static __int16)
didn't allow a second instance of the filter in the same chain; cause it repeat the
first input parameters; are still present in two cases in DGDecode.dll.
In fact what I've done was completely change the way to pass the parameters

Another question it has not parameters controls.In some cases it can output corropted
images.I've also included that.

There also an important feature which I've also pointed
in this thread http://forum.doom9.org/showthread.php?t=71452
more than a year ago.Nobody answered.
Finally I've believe that when MarcFd tested LumaFilter was in time when there
was a SeparateFieldsYV12 in MPEG2Dec3 0.94 which worked creating two real new frames
in different memory areas.SeparateFields the internal one just redifine the size of pitches
and readjust the pointers.That is why any filter built over frame area (pitch*height)
will not work correctly.To avoid that I've include row loops when vi.IsFieldBased()=true
By now is an easy solution but I'm not sure if it is the best.
Anyway for my own use I've modified a little bit SeparateFieldsYV12 as an idependant plugin
If anyone is interested to bring it up just ask.

Obviously the -2 parameters as default you've already corrected

And finally I've made separate assemblers codes for SSE2 ISSE and MMX
where I've added some optimizations.
There is a lot of duplicated code and I hope as always you can be generous
in your opinions. This plugin was renamed as you can see for one I've found more
appropiate and to avoid conflicts

Many would ask why all this; just to improve perfomance and in some of my machines
more than 50%.
***********************************************************************************
LumaYV12 plugin for Avisynth 2.5 by ARDA.

LumaYV12 is a new version of LumaFilter included in MPEG2DEC3.dll by MarcFD.

It assumes [0->255] YUV range, and not CCIR 601 [16->235].
Use limiter() afterwards if you think you need it.

Syntax: LumaYV12(lumoff=param,lumgain=param) or LumaYV12(param,param)

WARNING!!WARNING!!

lumoff=-256 to 256 (integer) ; default 0 and will do nothing.
lumgain=0 to 2.0 (float) ; 1.0 is default and will do nothing.
************************************************************************************
WARNING!!WARNING!!

This plugins was done for my own use trying to achieve a better perfomance
in my pentium4 cpu; I've also included Integer SSE and MMX routines but
perfomance was not tested.
It must run on any SSE2 machine but perfomance is not guaranteed.

I've also included row routines to avoid first line problem after separatefields.
Seems there is not real two frames, but a redefintion of pointers and pitches.
Maybe there's another way to avoid that problem but I couldn't find an easier one.
Row loops will be here till SeparateFields could be modified.

Actually they are exercises; so there are a lot of unnecessary duplicated code.
There must be a lot of spanglish variable names; my apologizes for that.

Please be patient!.-
I'm an eternal newbie coder.-
************************************************************************************


download
http://ardaversions.iespana.es/lumayv12.7z

As always I hope this can be useful.
ARDA

[Guest]

The way the variables were declared (__declspec(align(8)) static __int16) didn't allow a second instance of the filter in the same chain; cause it repeat the first input parameters; are still present in two cases in DGDecode.dll. Agreed. Thanks for pointing it out.

Another question it has not parameters controls.In some cases it can output corropted images. Range checking? Fair enough.

There also an important feature which I've also pointed
in this thread... I've duplicated this and it is a bad problem. I'm not sure I like the way you've fixed it. I have to think about it.

And finally I've made separate assemblers codes for SSE2 ISSE and MMX where I've added some optimizations. That sounds useful.

I'll test your filter version and if it appears OK, I'll replace the existing broken one with it. Thanks for your contribution.

Also, if you could report your issues in the development thread prior to the release candidates appearing, it would be appreciated.

[Marius-the-Mad]

neuron2, thank you very much for MPEG 1 support. :) DGDecode is great and now it's even better. :)

I have a question connected to MPEG 1 support. DGDecode seems to output YV12 by default for MPEG 1. Is it intended behaviour ? If it is, does using upconv=true result in original YUY2 or is it the upsampled version ?

Sorry if I'm being silly, but I couldn't find anything about it. :(