DGMPGDec 1.2.0 beta 6

[Guest]

Here is the beta:

http://neuron2.net/fixd2v/dgmpgdec120b5.zip

Let's just say it fixes a lot of things and adds some cosmetic things.

This MPEG stuff is hard! I remember why I took a sabbatical now. :)

The starting codebase for DVD2AVI was a spaghetti bowl. I keep saying it needs a total rewrite, but Nic already tried that, I think.

[Cyberia]

Originally posted by neuron2
I remember why I took a sabbatical now. :)Don't even think about it. :eek:

The starting codebase for DVD2AVI was a spaghetti bowl. I keep saying it needs a total rewrite, but Nic already tried that, I think.
Is it any better now?

Still have the issue with the seek bar where it doesn't display the right frame if you move the bar too fast.

Load a vob. Grab the bar and snap it to the right all the way, fast. Repeat.

Pretty soon, you'll have a situation where the bar is at the endpoint, but the displayed frame is from much earlier. If you click 'Next GOP' it advances from the displayed frame.

[Guest]

It's not worth fixing that. I'd rather rewrite the entire navigation system.

What I'm interested in now are what people have real practical problems with. Obviously audio sync is a biggie, and now I am realizing that crappy transport streams are causing people a lot of headaches. I'm thinking we need DGTransportFix. It would recover the PCR and then with that stable timebase, it would expect a video frame at regular intervals of PTS corresponding to the frame rate. If one is missing, we either insert one or cut audio to match. Similarly for missing audio. Of course the hard part is the inserting of video and audio. You'd need to reencode the MPEG locally, but that is not terribly difficult.

That's what I'm thinking about.

[zeus163]

First off, thanks for continuing to work on this great program. It is one of my most used programs.

I seem to have a slight problem in regards to .ts streams. I capture my streams via the firewire out of my Motorola box onto my computer. When I record off Discovery HD I can run into problems with the streams in Project X, but that's not too big of a deal as I just edit out commercials with HDTV2MPEG. Well, in HDTV2MPEG it is reporting the streams as video 22 and audio as 20. When I input these into DGMPEG it states that it can't find anything. When I have DGMPEG try and find the pids it finds 11 and 14. It seems to work fine until the very end of creating a project when it brings up an error that states that it can't find audio or video data and that I need to recheck my pids. An AC3 file is still created however. I just ran it again and it created an ac3 file with a different offset then the first time I ran it through with the same erro at the end. Now I'm confused. I just ran it a third time with the same error, but now two of the ac3 files have the same offset. I'm going to try and encode this overnight and see if it all works out.

Whew...I hope that makes sense.

[Guest]

@zeus163

I can help with it only if you put a failing source stream on my ftp site and then tell me it is there:

ftp.neuron2.net
user: guest@neuron2.net (enter it just like that)
password: guest

[zeus163]

Thanks Neuron2

I have to head out to work now. I'll up a piece of it when I get back. I'll edit out a small clip with HDTV2MPEG2 and start it later.

thanks!

[Nic]

Mine turned into a spaghetti bowl too...You start writing things modular, and then realise you need to do them all together....

Maybe you and me should start from scratch one day...one day ;)

Keep it up,
-Nic

[SoulSmasher]

thanks neuron 2,
hope this should solve some of my encoding problems (on my thread)

[Cyberia]

I can't find any real-world issues with this build.

The trackbar has MUCH better resolution and it looks like Preview mode is about 5-10% faster. Project time remains the same though.

Nice job!!!

[TEB]

@Neuron2: Great build but...
1. Sometimes when u add a TS file and the "cant find pids" popup comes along after a few seconds.. But if u are to fast and click on detect pids BEFORE this popup comes it hangs the application.. I was able to do this with random TS yesterday...
2. Some HDV file (like the one u got from me) says cant find PMT/PAT, so no pids are selectable..

@teb

[Guest]

@TEB

1. It shouldn't hang. I'll try to duplicate it.

2. If DGIndex says there is no usable PAT/PMT, then there isn't. You can always use Raw PID Detection to set the PIDs.

[loni_blues]

Hi neuron2,

First, thanks for your great work!

There is a problem when saving a d2v file with beta 3. The file is left unfinished with the last couple of lines missing (checked with 1.1.0).

A doubt: I thought timestamp stood for the part of the movie being analysed, but at around half of the movie I am processing it starts from 0 again and ends with a value of 53 minutes for a 2 hour movie. Does that have to with the number of audio tracks?

Regards

[Guest]

I have an idea about the unfinished business. I'll look at it tonight. Can you tell me if it was a VOB, a transport stream, or an elementary stream?

I don't know anything about the timestamps. I'll have a look at that too.

[loni_blues]

It was a VOB. I noticed the timestamp thing in 1.1.0 as well.

Thanks, neuron2.

[CruNcher]

neuron2 i have the same problem only that it ends in an unrecognized exception error on load of the d2v via dgdecode also a vob file, it was created via the latest dgindex the new dgdecode works fine with old dgindex created d2vs

Edit: Dgindex.exe from beta2 works

[Guest]

OK, I got too clever for myself, apparently. I'll sort it out this weekend. Thanks for the feedback.

[zeus163]

@neuron2

I cut a piece of the .ts file that gave me that error message. I ran dgmpgdec on it and I could not recreate the error. Strange. So, I elected not to send the cut file to you.

Thanks for working on great programs like this for us!

[Boulder]

A bug I just found with 1.2.0b3 :

I tried to create a project from a movie, 4 VOBs. I added them normally and then chose "demux all tracks" and saved the project. The processing went fine until the point where it should have ended (that is, the end of the movie). Instead, it kept going, and for some weird reason the resulting AC3 track became huge, it was ~3GB by the time I closed DGIndex :eek: Choosing one track and demuxing only that didn't help either.

I ripped the DVD with DVD Decrypter, stream processing enabled, one track left out.

DGIndex 1.1.0 works fine.

[Guest]

Yes, I'm aware of it. I'll release a fix today or tomorrow.

Thank you.

[Guest]

Here's beta 4:

http://neuron2.net/fixd2v/dgmpgdec120b4.zip

Hopefully (need smiley with fingers crossed), it fixes the reported problems. It can demux video from VOBs now too (transport and PVA coming). It also fixes the bug in deringing that Milan pointed out.

[off to the swimming pool now]

[TEB]

@Neuron2: Works good. Tho i get a VIDEO-ERROR in the gui on a file here that i know should work ok. Its on the ftp as zzzz_remuxed.spts.ts

[gpower2]

Great job! It is really nice that you continue this project!
Can you provide the source from 1.2.0 so I can compile it with ICL 8.1?
I "discovered" this ability lately, meaning I can compile with ICL 8.1 and optimize the binaries for my P4 with SSE2, and I saw great results!
I will soon provide some of my binaries in this forum (XviD 1.1, DGdecode, DGindex, DGbob, avisynth, etc...) for all us P4 users!

Thanks again!

[Guest]

Originally posted by gpower2
Great job! It is really nice that you continue this project!
Can you provide the source from 1.2.0 so I can compile it with ICL 8.1? I don't release source for betas.

I "discovered" this ability lately, meaning I can compile with ICL 8.1 and optimize the binaries for my P4 with SSE2, and I saw great results!
I will soon provide some of my binaries in this forum (XviD 1.1, DGdecode, DGindex, DGbob, avisynth, etc...) for all us P4 users!
I prefer that you don't do that for my tools. If you insist, please use different names for the executables to avoid versioning nightmares. Thank you.

[marcellus]

Hi Neuron2, many thanks for your precious tool.
I just sent on your ftp a small m2v file (it's in an archive named long_gop.7z) with a problem. This particular fragment (SVCD PAL) I admit that probably is not mpeg2 compliant. I used ffdshow to encode it and the thing that makes it so special is that I set maximum I frame interval to 500. Libavcodec gives pretty good results with long gops (anyway much better than with a GOP of 15, especially at low bitrates). Funny enough, my standalone have no problem playing that kind of files (only seeking is slow) and all mpeg2 soft players play it also.

DGIndex again have no problem playing or previewing it but when I try to save the d2v file it simply vanishes after saving a 0 bit d2v file.

All versions of DGIndex that I tried show the same behaviour (including latest beta).
Bye

[Guest]

I support a maximum GOP of 499 pictures. What a coincidence! Try 499 and report back. Thank you.

EDIT: The file you uploaded does not contain even one sequence header. It is therefore invisible as far as MPEG2 decoding is concerned. You may have cut it too small.

Don't you think 500-picture GOPs is a bit of overkill?

[Guest]

Here is beta 5. The only thing it fixes is an important bug in indexing that could cause the first GOP address to be wrong in some circumstances (side effect of the changes to support dynamic quant matrices). Unfortunately, this affects 1.1.0 as well. :(

http://neuron2.net/fixd2v/dgmpgdec120b5.zip

I recommend using this version over 1.1.0.

[mp34ever]

i have been playing with beta 5 and i have golmember and a anime movie ripped of dvd to my computer in vob format and when i load the anime movie i go preview it looks interlaced and it says it is interlaced in preview window.But when i load goldmember in the preview windows it says it is interlaced but when i look at the frames it is progressive no lines.It looks like it has been deinterlaced really good.The goldmember vobs are actually interlaced because i checked them in virtualdub mod and i needed to deinterlace it?.

[Guest]

You were quick. Download beta5 again because I had a bad version up for about 5 minutes.

Still, it wouldn't explain your issue.

The preview window shows only the first I frame of a GOP. To be sure, save project with field operation NONE and then serve to VirtualDub. If you don't see what you expect, please post a portion of the D2V file.

[Guest]

Originally posted by TEB
@Neuron2: Works good. Tho i get a VIDEO-ERROR in the gui on a file here that i know should work ok. Its on the ftp as zzzz_remuxed.spts.ts Are you sure? I don't see it there.

[Guest]

Originally posted by loni_blues
A doubt: I thought timestamp stood for the part of the movie being analysed, but at around half of the movie I am processing it starts from 0 again and ends with a value of 53 minutes for a 2 hour movie. Does that have to with the number of audio tracks? I had a look at this. The "timestamp" is just the audio PTS converted to hours/minutes/seconds. That means it can wrap around and is actually not very useful.

[mp34ever]

i redownloaded beta 5 and i am going to try what you said but this happened once before i had the matrix 1 on dvd and i ripped it and made a avs script but i wasn't deinterlacing it and vdub showed it progressive so i thought okay and converted to xvid .Once it finished
i viewd it in divx player and vdub and the whole thing was progressive apart from the start wher it had a paramount logo for first 10 seconds and that was interlaced.It was a official version not a asian rip.wierd.

[mp34ever]

i also had a test clip which i encoded in tmpgenc with a long gop not over 400 and powerdvd played it fine but dvd2avi only showed 2 frames and a green one and it was over a 60 seconds long.I remember ages ago with dvd2avi (probably fixed by now but hey) when selecting fromthe idct selection i selcted 1180 reference (something like that)and it created the little square artifacts not like macroblocks.

[Guest]

@mp34ever

You're confusing me. Please stay focused on the present. Do you have a problem or not? If so, please put a clip on my server and explain how to duplicate the problem.

ftp.neuron2.net
user: guest@neuron2.net (enter it just like that)
pwd: guest

[TEB]

@Neuron2: Let me refrase it (tested on beta5).. Its under the Status:Info:Videoerror.

[Guest]

Originally posted by TEB
@Neuron2: Let me refrase it (tested on beta5).. Its under the Status:Info:Videoerror. Ha ha. I meant I don't see the file on the ftp site!

[Guest]

@TEB

There's a long-standing bug in the motion vector code in the whole DVD2AVI line of code. I just got a program stream that does it from another person. It fails in all versions of DVD2AVI all the day back to day 1. It's a problem with motion vector decoding. I will try to fix it.

[alfixdvd]

File d2v version 1.1.0 and 1.2.0 beta 5 are different at values 7 0 xxxxxxx in Gop Lines.

This is related with "indexing that could cause the first GOP address to be wrong in some circumstances " ?

Why if I examine mpegs generated with version 1.1.0 with M2Editpro it says thats's OK. ?

best regards
alfixdvd

[Guest]

Originally posted by alfixdvd
File d2v version 1.1.0 and 1.2.0 beta 5 are different at values 7 0 xxxxxxx in Gop Lines.

This is related with "indexing that could cause the first GOP address to be wrong in some circumstances " ?

Why if I examine mpegs generated with version 1.1.0 with M2Editpro it says thats's OK. ?
Yes, the indexing was sometimes wrong.

The bug affects only the indexing (and therefore also DGDecode). If you are talking about video demux'ed by DGIndex, it is not affected by this bug.

[alfixdvd]

@Neuron2: Thank you for your answer.

best regards.
alfixdvd

[marcellus]

Hi Neuron2,

Don't you think 500-picture GOPs is a bit of overkill?

Yes, maybe, but I like to push things to limit (guess what GOP size I would use if ffdshow slider could reach a 1000 I frame interval setting). Anyway, this is how I discovered the DGIndex issue, because I was trying to compare quality through SSIM and PSNR at various GOP sizes.

Try 499 and report back.


The file you uploaded does not contain even one sequence header. It is therefore invisible as far as MPEG2 decoding is concerned. You may have cut it too small.

I've uploaded another sample made with a GOP setting of 499, it's in 499_gop.7z. No change, DGIndex still vanishes. I lowered the bitrate (I'm on dial-up) but I cut it longer. It's cut with mpeg2-schnitt from a 2 pass encoding made on 20000 frames (set at 500 kbps) and unmodified in any way (I think that the one I sent you previously was passed through Re-Stream), so if you see anything unusual in it (like the missing sequence header) it's probably a ffdshow/libavcodec issue.
Bye
Edit: I checked and it seems that my upload failed (my line dropped). I will send it again.

[Wilbert]

@Neuron2,

I've a small request :) Is it possible to pass the colorimetry (ie I709, FCC, I470, S170 or S240) through as global variable, such that ColorMatrix can use it:

ColorMatrix(CoefHints=true)

[Cyberia]

Originally posted by Wilbert
@Neuron2,

I've a small request :) Is it possible to pass the colorimetry (ie I709, FCC, I470, S170 or S240) through as global variable, such that ColorMatrix can use it:

ColorMatrix(CoefHints=true)
Wouldn't this be included in this Dev list item:


DGDecode Items
---------------
3) Allow "Info()" to be used in AviSynth for conditional filtering


What is ColorMatrix exactly?

[Wilbert]

Wouldn't this be included in this Dev list item:
More or less. The colorimetry parameter is sufficient for me :)

What is ColorMatrix exactly?
http://forum.doom9.org/showthread.php?s=&threadid=82217

summary:
There are multiple way to do a YUV<->RGB conversion, ie using different coefficients (the conversion itself is linear). Some streams are created uses Rec.709 coefficients (many dvds), others like DivX/XviD are created using Rec.601 coefficients.

[Guest]

Originally posted by Wilbert
I've a small request :) Is it possible to pass the colorimetry (ie I709, FCC, I470, S170 or S240) through as global variable, such that ColorMatrix can use it:

ColorMatrix(CoefHints=true) You mean as hints in the video ala Decomb? If so, sure. I'll define an "API" and add it for you.

[Wilbert]

You mean as hints in the video ala Decomb? If so, sure. I'll define an "API" and add it for you.
Yup. That would be very cool! Thx.

[tritical]

Um, wouldn't using env->SetVar() and env->GetVar be more flexible? Using the method of hinting from decomb would mean colormatrix would have to be called immediately after mpeg2source() or at least before any filter that modifies the first 64 bytes of the image, and means you could not use it on interlaced material using separatefields().colormatrix().weave(). However, it would be nice to be able to do this and also to be able to use it after resizing I think? The setvar, getvar method would allow this and still be able to work correctly with multiple instances of mpeg2source() in one script.

[Guest]

We want to handle per-frame data, so we can pass the interlaced/progressive indication, etc. Colorimetry would be signaled the same way. I can't think how to use SetVar/GetVar for per-frame data. This has been discussed extensively and the consensus is that Avisynth would support per-frame associated data. Until then, well, hints is all I can think of.

[len0x]

Isn't colorimetry a global setting for the mpeg source? (i.e. it doesn't change from frame to frame within the same source)

[Guest]

It's in the sequence display extension, so it can change with every sequence. Anyway, I prefer to have one mechanism and since I need the per-frame data for interlaced/progressive, field/frame, etc., I might as well use it for the colorimetry as well.

[tritical]

Actually, you can use setvar and getvar to pass per frame values using avsvalue arrays. It isn't super nice and clean, but it is not too difficult. You set up an array that contains the frame # and then the next entry could be the value for that frame # you want to pass. The array wouldn't even need to be equal to the total number of frames in the clip, but only long enough to get past any caching and look ahead (200-400 would be more then sufficient). Then the next filter in the chain searches the array for the corresponding frame number entry and gets the data it needs.