DVD2AVIdg/MPEG2DEC3dg Version 1.2.0

[SILICON]

I ignore if are one stupid request.

I see that the memory bandwid cut down speed in avisynth.
If you can pass the crop values to DVD2AVI, the MPEG2DEC3 decode less data and send less data to avisynth.

Less data more speed, isn't it?

Request:
If above not are stupid, pass the crop values in command line.

[Guest]

Originally posted by Moitah
Let me know if you need me to upload it somewhere.Yes, please upload it and notify me where I can get it.

Do it quick so I can look at it this evening. Thanks.

[Trahald]

is it possible for more information to be included. can field dominance be derived from the d2v? the 3:2 break is reported during pure video when its not a suprise that would be the case.

[Guest]

@trahald

ParseD2V tells you the dominance. If the first flags digit of the clip is 2 or 3 or 12 or 13 (TFF set) it is top field first, else it is bottom field first.

I didn't get your second point. It's true but so what?

@Moitah

I love you, man. That clip is exposing hidden little nooks and crannies of the code and will finally expose the reason for the strange LBA-1 address kludge. I'm getting close but it is very complicated.

[Trahald]

Originally posted by neuron2
@trahald

ParseD2V tells you the dominance. If the first flags digit of the clip is 2 or 3 or 12 or 13 (TFF set) it is top field first, else it is bottom field first.

I didn't get your second point. It's true but so what?



I was at work and didnt finish my thought. I guess I was just saying instead of that message something else can go there. Maybe the field dominance or something. At least that what I was thinking. However since you provided how to determine dominance that doesnt seem to be necessary. ...beyond that it is harmless and is just fine. :)

And thank you for the info.

[Cyberia]

@neuron2 - Actually, the VOB slicing enhancement might be a good one to get, because, really it's a debugging option for you.

[Guest]

Here is a "fix" for Moitah's issue. Well, it works with his clip anyway. :)

http://neuron2.net/fixd2v/decodefix120RC3.zip

Now, permit me to make a few remarks.

This clip of Moitah has exposed an ugly underbelly of DVD2AVI/MPEG2DEC3. Notice I omitted the "dg" part because this underbelly is in the basic stream parsing code, which I have not touched. I have touched only the MPEG2 elementary stream decoding. It is now clear that the layered stream parsing is a giant kludge and not provably correct. It may work for most streams but as Moitah showed, a given clip with slightly unusual pack allocations can cause difficulties. ES can be arbitrarily packed in PES but the code does not allow for this, instead making some assumptions that are usually true for DVDs. But throw an arbitrary program stream at it and...good luck!

So I have added yet another kludge on top of the existing kludges to address Moitah's issue. That is what DVD2AVI is! But it cannot go on. It's time for a major departure. Therefore, I will start a totally new development, with the goal of having provably correct decoding at every layer: elementary, PES, program, transport, etc. Furthermore, DVD2AVI did not begin from the ground up with a view to random navigation (and especially via Avisynth). That should be a starting premise of the new development.

So, I will start a new thread for this development. Does it belong in this forum? Please advise.

[fccHandler]

Originally posted by neuron2
But it cannot go on.
Sorry to hear about the ugly underbelly. I've been silently waiting in the wings for a long time for DVD2AVIdg's development to stabilize, because I wanted to try the code in VirtualDub-MPEG2. My idea was to replace the parser with a built-in DVD2AVI (minus its decoder), so the "parsing" stage would build something like a D2V in memory. (Actually it does something very similar already.) Then I could replace the decoder with Mpeg2Dec code, only changing the interface to be compatible with VirtualDub's classes.

Right now the MPEG parser is a mess of spaghetti, but all it really lacks in functionality is rff/tff support. I've tried many times to implement that without much luck. One time I thought I almost had it working, then somehow it fell apart again.

You seem to have a lot of time on your hands right now (more than me, anyway), and I think you are the only MPEG-2 developer I know who could save this code, if you're interested.

Thoughts?

[Cyberia]

Are you just going to rip out the parsing mess and replace it with better code? Or start entirely over from the ground up? If the latter, then that would be a good time to implement the new DGMPGDec/DGParse/etc naming convention and versioning.

[RB]

Originally posted by neuron2
So, I will start a new thread for this development. Does it belong in this forum? Please advise.
Yes, I think it's best kept here. And good luck, I think a complete rewrite was long overdue, thanks for taking the challenge! Are you going to rewrite it in a more object oriented way?

[vispgraedde]

Good luck with the rewrite as it seems to be needed...

And I just found some bug with RC2 that is there with RC3 as well...

When I open the d2v in tmpg (making sure that it use the correct vfp version just in case), I often have to click on options like inverse telecine and source range twice before I get the video...
And when I was about to encode, I got some error messages:

Read error occured at address 00F2B7D3 of module 'mpeg2dec3dg.dll' with 04B2E000. (RC2 Raw Frames)
Read error occured at address 00FB3E42 of module 'mpeg2dec3dg.dll' with 049DE0E8. (RC3 Raw Frames)
Read error occured at address 00FB3E42 of module 'mpeg2dec3dg.dll' with 05715D00. (RC3 None)

[Guest]

Actually, on further analysis it appears that the parsing layer is fine. The problem is purely with random navigation. I'll start a new thread on this and my plans. This thread will continue with version 1.2.0 development.

Here's one case where random access fails for program streams: You want frame 17 and the D2V file gives you the LBA that contains the I frame for the GOP that frame 17 is in. The LBA indexes are based off the location of the I frame so that means that the packet header could be in the previous LBA. If we miss the packet header we can't pick up the I frame and we parse forward to the next packet which gives us the wrong frame.

This is what I meant when I said that the design was not made from the ground up for random frame access. When you lay a higher layer over the elementary stream indexing, you have to change to indexing at the higher layer, in other words the packets should be indexed and not the I frames.

Whether the existing code can be rescued for fully reliable random access remains to be seen.

[Guest]

Originally posted by vispgraedde
When I open the d2v in tmpg (making sure that it use the correct vfp version just in case), I often have to click on options like inverse telecine and source range twice before I get the video...
And when I was about to encode, I got some error messages:
You are making a 1.76 style D2V and then using VFAPIdg to serve to TMPGEnc. Is that correct?

[vispgraedde]

This is using dg style d2v's in all cases. Not 1.76. And using VFAPIdg (DVD2AVIdg.vfp) for the frameserving.

Edit:
I also remade the d2v's with RC3 just in case.
If it matters, the original m2v is >4GB on NTFS partition in WinXP.

[Guest]

What happens if you serve it into VirtualDub?

[vispgraedde]

In vdub, I get the same artifacts as I can see in tmpg for the first two frames, the first time they are shown.
All around the video there is supposed to be a black border.
Instead of the black border for the first two frames, I have a green border that seems to consist of 16x16 pixel blocks, and in some places in the border, it's only 8x8 pixel blocks of green.
This is only for the first two frames that I step through.
The video does not load every time either. Sometimes I only get black through the whole thing, but video is back after I reload the pseudoavi.

[vispgraedde]

The beginning of the d2v itself:

DVD2AVIProjectFile1
1
82 E:\DYRL\VTS_01_1 - 0xE0 - Video - MPEG-2 - 720x480 (NTSC) - 16~9 - Letterboxed.M2V

Stream_Type=0,0,0
iDCT_Algorithm=2 (1:MMX 2:SSEMMX 3:FPU 4:REF 5:SSE2MMX)
YUVRGB_Scale=1 (0:TVScale 1:PCScale)
Luminance_Filter=0,0 (Gamma, Offset)
Picture_Size=0,0,0,0,0,0 (ClipLeft, ClipRight, ClipTop, ClipBottom)
Aspect_Ratio=16:9
Field_Operation=0 (0:None 1:ForcedFILM 2:SwapOrder 3:RawFrames)
Frame_Rate=29970
Location=0,0,0,2DD1E0

7 0 0 0 0 12 12 12 12 12 12 12 12 12 12 13 10
7 0 D6 0 0 1 2 13 10 11 12 13 10 11 12 13 10
7 0 1E7 0 0 1 2 13 10 11 12 13 10 11 12 13 10



--
The mpeg2 is hybrid encoded (part film, part ntsc).

[vispgraedde]

To eliminate VFAPI as the problem, I just tried through avisynth as well, with the same artifacts. Green border for the first two frames but this time, the green border is always present for those frames instead of disappearing after stepping back.

[Guest]

I need your clip that fails. Try to cut it to a decent size and post a link where I can download it. Thank you.

EDIT: Forgot to ask... This clip does play in Media Player, for example, i.e., it's a known good stream?

[Guest]

Here is RC4 which fixes the issue with postprocessing settings not being individually settable for multiple instances of MPEG2DEC3 in a script, and fixes a stupid memory overwrite that may be the reason for vispgraedde's problem. Do not use RC3!

http://neuron2.net/fixd2v/decodefix120RC4.zip

[vispgraedde]

RC4 fixed the green border problem, but not the other problem I experience.

And now when I open the d2v in tmpg, or sometimes when I try to encode, I get error messages like:

Write error occured at address 00FAB5E7 of module 'mpeg2dec3dg.dll' with 0200E080. (emphasis mine)

I will pm the URL to where I stored a small clip for which I experience this problem.

I have used this stream through avisynth without any visible problems before.

[Guest]

What do you mean by opening D2Vs in TMPGEnc? I thought you were using VFAPI?

If you want to read a D2V with existing apps, you need to use 1.76 style.

[Cyberia]

Basic file stuff:

Please prompt if the new project will erase an existing one.

Please popup a reason why a file save would fail. (eg: try to save a project that will overwrite an existing READ-ONLY file.)

FYI: I am leaving tonight for a ten day vacation in Maui. I will try to read the forum, I will have a laptop, but I will certainly not be at full strength for the duration. I will drink a mai-tai for each of you, and two for Don. Aloha.

[Guest]

Originally posted by Cyberia
Please prompt if the new project will erase an existing one.

Please popup a reason why a file save would fail. (eg: try to save a project that will overwrite an existing READ-ONLY file.) Done! Thanks for the suggestions.

FYI: I am leaving tonight for a ten day vacation in Maui. I will try to read the forum, I will have a laptop, but I will certainly not be at full strength for the duration. I will drink a mai-tai for each of you, and two for Don. Aloha. Thanks and have a great time!

[vispgraedde]

Originally posted by neuron2
What do you mean by opening D2Vs in TMPGEnc? I thought you were using VFAPI?

If you want to read a D2V with existing apps, you need to use 1.76 style.

I thought the whole point of VFAPI was that the apps does not need to have a native d2v parser if it goes through the vfp that dvd2avi registers.
And therefore you don't need to keep track of what file version you use, as long as the vfapi plugin matches the file you want to read.

[Guest]

Originally posted by vispgraedde
I thought the whole point of VFAPI was that the apps does not need to have a native d2v parser if it goes through the vfp that dvd2avi registers.
Yeah, I knew that. :p

So, I found and fixed the problem and PMed you a link to the fixed DLL. When you give the thumbs up I'll make the next release. I'm sensing 1.2.0 is nearing release.

[vispgraedde]

While you are at it, can you change the version string that is reported through vfapi as well? (For the 1.20 release)
The one that shows up in Options->Environmental setting->VFAPI plug-in list in tmpg.

[Guest]

Change it to what?

EDIT: I changed it to "DVD2AVIdg 1.2.0 D2V Reader".

[vispgraedde]

Edit:
nm, I see you edited your post :p

[zettai]

Hey neuron2,

I don't mean to be a pest but I was wondering if you are going to work on the Progressive/Interlaced hints before starting on the new parser. I guess this goes for any of the worklist items - are they to be addressed before trying to write a new parser code?

I'm all for a better parser, of course, but I'd really love theupsampling in avisynth to be easier too :)I'd rather not have to use vfapi for it :P

[Guest]

OK, here is RC5. Five is my lucky number so maybe this one will be the ticket.

http://neuron2.net/fixd2v/decodefix120RC5.zip

This one adds the file checking that Cyberia suggested and fixes the VFAPI problem with TMPGEnc that vispgraedde reported.

[zettai]

One thing I just thought of (when thinking of these upsampling issues) is the post processing options of mpeg2dec3.

Surely the option ipp=true/false isn't necessary... can't the post-processing base it on the current frame type?

Just a suggestion - I'm not the only one who uses hybrid footage, honest :P

[RB]

Donald, did you implement anything in terms of Mpeg2Dec3dg internal upsampling to YUY2 depending on progressive_frame flag?

[Guest]

@zettai

Yes.

@RB

Not yet. Version 1.3.0 will have transport streams/HD support and upsampling fixes (and maybe more). I want to get 1.2.0 stable out the door.

[jimmy basushi]

wow neuron2, you have pushed out so many new versions of dvd2avi lately i just wanted to thank you for spending your time working on such a necessary dvd tool.

[Guest]

Thanks. But you ain't seen nothing yet. :cool:

[JohnMK]

Donald Graft rocks my world. :thanks:

[Cyberia]

Aloha from Maui! I just wanted to thank Don for the hard work.

Thanks. Time for a mai-tai.
Cyb

[Cyberia]

OK, I have no vobs here at all, so I tried to open a Word doc with RC5. Well, bad things happened when I saved the project.

You may want to fail more gracefully if its at all posible. This could wait for 1.3 however.

[Guest]

Originally posted by Cyberia
This could wait for 1.3 however. Oh, thanks, Cyb. That's very considerate of you.

I've just totally rewriitten the indexing support. This LBA-1 crap just doesn't cut it, ya know. Drink a mai tai or two in my honor, dude. Then you won't care if you can't decode a Word doc.

Tomorrow I'm going to rewrite all the trackbar navigation stuff. It sucks.

[DDogg]

Donald, perhaps not appropriate for this thread. Delete/move if needed to keep order.

I'm a little confused / more than normal.
Source "GoodFences", Showtime made for TV movie on DVD. RC5 Shows BFF and from the D2V below seems 100% USDA interlaced.

So, I thought I would check the BFF just to be sure and used the 'AssumeTFF().SeparateFields()trick in VDub. It showed TFF motion. Seems like I remember you saying not to depend upon D2V correctly identifying field precedence but I thought I would mention it since you are down in the guts of the beast now.

Dvd2avi shows this as an 100 interlaced source, yet it shows the 3 good, 2 combed pattern when viewed in VDub. Is this as you would expect from the pattern below? The rest of the file is just like the snip shown.


Stream_Type=1,0,0
iDCT_Algorithm=2 (1:MMX 2:SSEMMX 3:FPU 4:REF 5:SSE2MMX)
YUVRGB_Scale=1 (0:TVScale 1:PCScale)
Luminance_Filter=0,0 (Gamma, Offset)
Picture_Size=0,0,0,0,0,0 (ClipLeft, ClipRight, ClipTop, ClipBottom)
Aspect_Ratio=4:3
Field_Operation=0 (0:None 1:ForcedFILM 2:SwapOrder 3:RawFrames)
Frame_Rate=29970
Location=0,0,6,181C7

7 0 1 1 1 10 10 10 10 10 10 10 10 10 10 10 10 10
7 0 27 1 1 12 12 12 12 12 12 12 12 12 12 12 12 12
7 0 ED 1 1 12 12 12 12 12 12 12 12 12 12 12 12 12
7 0 1AE 1 1 12 12 12 12 12 12 12 12 12 12 12 12 12
7 0 278 1 1 12 12 12 12 12 12 12 12 12 12 12 12 12
7 0 35B 1 1 12 12 12 12 12 12 12 12 12 12 12 12 12
7 0 422 1 1 12 12 12 12 12 12 12 12 12 12 12 12 12
7 0 4D0 1 1 12 12 12 12 12 12 12 12 12 12 12 12 12
7 0 58E 1 1 12 12 12 12 12 12 12 12 12 12 12 12 12
7 0 65B 1 1 12 12 12 12 12 12 12 12 12 12 12 12 12
7 0 71C 1 1 12 12 12 12 12 12 12 12 12 12 12 12 12
7 0 7DD 1 1 12 12 12 12 12 12 12 12 12 12 12 12 12
7 0 8A4 1 1 12 12 12 12 12 12 12 12 12 12 12 12 12
7 0 968 1 1 12 12 12 12 12 12 12 12 12 12 12 12 12
<snip>
7 6 17E4B 2 7 12 12 12 12 12 12 12 12 12 12 12 12 12
7 6 17F0C 2 7 12 12 12 12 12 12 12 12 12 12 12 12 12
7 6 17FD6 2 7 12 12 12 12 12 12 12 12 12 12 12 12 12
7 6 18097 2 7 12 12 12 12 12 12 12 12 12 12 12 12 12
7 6 18161 2 7 12 12 12 12 12 12
7 6 181C2 2 8 10 10 10 10 10 10 10 10 10 10 10 10 10 9

FINISHED 0.00% FILM

[Cyberia]

Oh, thanks, Cyb. That's very considerate of you.

Not sure if I detect sarcasm there or not. Part of my job (tech support) is to imagine how morons could break things. I realize you shouldn't open a Word doc with DVD2AVI, but... the same could happen with a corrupted d2v file.

I so piss off the developers at work by thinking of things you shouldn't do anyway, but the program should never ever GPF...

Another mai-tai...check!

ps: I am so clever... I rented Kill Bill and downloaded Smartripper, so I have a VOB now!

[len0x]

I seem to have missed all the new development of DVD2AVI recently, but having looked through the change log I haven't found that one old bug is fixed: basically it seems that on old dual (say P2-P3) systems when started from CLI DVD2AVIdg 1.0.0 crashes (only from CLI). Any ideas ?

Thanks.

[zettai]

Have you tried setting the processor tag in CLI?

[len0x]

Originally posted by zettai
Have you tried setting the processor tag in CLI?

IDCT you mean ? It's always set to SSE MMX... Probably there are still some threads which are not launched the same way from GUI and CLI...

[Guest]

Originally posted by DDogg
I'm a little confused / more than normal.
Source "GoodFences", Showtime made for TV movie on DVD. RC5 Shows BFF and from the D2V below seems 100% USDA interlaced.

So, I thought I would check the BFF just to be sure and used the 'AssumeTFF().SeparateFields()trick in VDub. It showed TFF motion. Look at the first two and the last two GOP lines in your post. They both have illegal transitions! E.g., in the last two lines it transitions from 2 to 0. That changes the field order. Try running Fixd2v on it and things should become coherent again.

You can't change the field order like that. The display device has to receive a strictly alternate sequence of top followed by bottom fields. 2 followed by 0 specifies: show a top field of A, show a bottom field of A, show a bottom field of B, show a top field of B. That's not an appropriate thing to be doing. The poor widdle display will get seriously disturbed.

Dvd2avi shows this as an 100 interlaced source, yet it shows the 3 good, 2 combed pattern when viewed in VDub. Is this as you would expect from the pattern below? The rest of the file is just like the snip shown. Then it's what we call hard telecining. The pulldown is applied prior to MPEG encoding and RFF flags are not used.

[Guest]

Originally posted by Cyberia
the program should never ever GPF... Right on. I have to agree with that.

[Guest]

Originally posted by len0x
I seem to have missed all the new development of DVD2AVI recently, but having looked through the change log I haven't found that one old bug is fixed: basically it seems that on old dual (say P2-P3) systems when started from CLI DVD2AVIdg 1.0.0 crashes (only from CLI). Any ideas ? Do you have the setup to duplicate this? If so, I can work with you by sending you debug loads.

[len0x]

Originally posted by neuron2
Do you have the setup to duplicate this? If so, I can work with you by sending you debug loads.

I don't, but lemme search the forum - may be ppl who had this problem are still around.

[vispgraedde]

I was bored and kinda grew horns here :devil: so I decided to see how dvd2avi handles a multiangle clip.

I produced 5 clips of 96 progressive frames, with pulldown flags, and produced a multiangle vob in Maestro from that.

My first find maybe isn't a bug per se, because I don't know too much about how multiangle work, but instead of getting 100% FILM, I only get 98.33% FILM (in this example).

Further, when I use forcefilm and go through avisynth, I get the correct 480 frames from this clip, but if I use none, I get 596 frames instead of the expected 600 frames. Maybe it's not a bug, but still :p (Encoding the clips with hard pulldown and using none produce the expected 600 frames)

Selecting what angle(s) you want in the output from the input, I have some ideas for that I haven't had time to think through fully yet, so I leave any talk about that alone for now...

I can provide said clip if you want it for testing :devil: ;)