does dvd2avi chop off frames?

[Guest]

I implemented the new random access scheme for mpeg2dec3, initially for non-forced film and no RFF handling. It works just fine and is surprisingly simple. It gets rid of all that obscure code and dumb GOPBuffer[] copying and so speeds thing up as a bonus. Following for the curious is the implementation. I need to add forced film and RFF handling and then we're done!

Note: Given a stream with an open first GOP, it still gets the number of frames right and seeking works, but (of course) you'll get garbage for the orphaned B frames. That seemed the best thing to do. If the first GOP is closed, everything is perfect.


void CMPEG2Decoder::Decode(DWORD frame, YV12PICT *dst)
{
unsigned int i, gop, count;

if (frame == prev_frame + 1)
{
// Decode next frame. No random access required.
Get_Hdr();
Decode_Picture(1, dst);
if (picture_structure!=FRAME_PICTURE)
{
Get_Hdr();
Decode_Picture(1, dst);
}
prev_frame = frame;
return;
}
else prev_frame = frame;

// Have to do random access.
// Determine the GOP that the requested frame is in.
for (gop = 0; gop < VF_GOPLimit-1; gop++)
{
if (frame >= GOPList[gop]->number && frame < GOPList[gop+1]->number)
{
break;
}
}

// Back off by one GOP if not GOP 0. This ensures enough frames will
// be decoded that the requested frame is guaranteed to be decodable.
if (gop) gop--;

// Calculate how many frames to decode.
count = frame - GOPList[gop]->number;

// Seek in the stream to the GOP to start decoding with.
File_Flag = GOPList[gop]->file;
_lseeki64(Infile[File_Flag], GOPList[gop]->position, SEEK_SET);
Initialize_Buffer();

// Start decoding. Stop when the requested frame is decoded.
Get_Hdr();
Decode_Picture(0, dst);
if (picture_structure != FRAME_PICTURE)
{
Get_Hdr();
Decode_Picture(0, dst);
}
Get_Hdr();
Decode_Picture(1, dst);
if (picture_structure != FRAME_PICTURE)
{
Get_Hdr();
Decode_Picture(1, dst);
}
for (i = 0; i < count; i++)
{
Get_Hdr();
Decode_Picture(1, dst);
if (picture_structure!=FRAME_PICTURE)
{
Get_Hdr();
Decode_Picture(1, dst);
}
}
}

[trbarry]

Donald -

Nice work!

- Tom

[Guest]

Originally posted by trbarry
Nice work!
Thanks. I need to grab your cute little binary GOP search fragment. I saw it after I coded the above.

[RB]

Wow, great news! Donald, thank you very much for taking care of this. I had already given up and thought that this issue just won't ever be resolved. Keep up the good work!

[trbarry]

Donald -

I notice you based your DVD2AVI fix off of V 1.77. Do you (anyone?) happen to know if that version can handle HDTV/ATSC streams?

- Tom

[Matthew]

Apologies for butting into the thread, I'd just like to confirm a simple way of checking that no frames have been dropped from during the film.

When there are 2 frames dropped at the beginning (vobedit reports delay of 0ms while DVD2AVI says -80ms so I know it's gonna happen) I use a simple avisynth script that puts 2 black frames in at the start (and I don't correct the delay).

Anyway, given my adjustment, is it correct to say that it's 100 percent certain that no frames have been lost during the film (start or end not included) if the number of frames in the encoded stream (as reported by CCE when avs is loaded) is 3 less than the number recorded in the original DVD's IFO file.

Tanks :)

[Guest]

Originally posted by trbarry
I notice you based your DVD2AVI fix off of V 1.77. Do you (anyone?) happen to know if that version can handle HDTV/ATSC streams?

I don't know but the changes can be easily ported to any version. If you provide me with a version that you need modified, I'll be happy to do it. First let's get the fix released and tested for 1.77.3 and then take it from there. I don't know where to find a release of DVD2AVI that supports HDTV/ATSC streams.

@all

I have just completed and tested RFF handling. It's working fine. I will implement forced film and then release the fix for testing.

I'm thinking of making a standalone mpeg2dec3 that can create the D2V file if it doesn't exist. Any interest in that?

[Guest]

Originally posted by Matthew
Anyway, given my adjustment, is it correct to say that it's 100 percent certain that no frames have been lost during the film (start or end not included) if the number of frames in the encoded stream (as reported by CCE when avs is loaded) is 3 less than the number recorded in the original DVD's IFO file.
DVD2AVI/mpeg2dec, even without my fix or your adjustment, never loses frames at any point other than the start or end.

[Guest]

I've completed the first beta of the full fix:

EDIT: Link updated to latest version.
http://neuron2.net/fixd2v/decodefix.html

Please post your test results and/or bug reports here.

I will make the source code available when testing is completed.

[int 21h]

Originally posted by neuron2
...
I'm thinking of making a standalone mpeg2dec3 that can create the D2V file if it doesn't exist. Any interest in that?..

That would be useful. Replacing forced film with Decomb's IVTC operations would be especially useful ;) Not sure if we'd see any speed increase from just using it in a script though.

[Guest]

The point would be to not need to have a compatible DVD2AVI trailing along. I don't see any advantage to putting Decomb in there. :-)

I would still allow appropriate options, like force film. But they would be passed as parameters in the mpeg2source() invocation.

[int 21h]

I can only speak for myself, but I usually use Mpeg2Dec and DVD2AVI for DVD-to-DVD operations, and I just create the .d2v and use DVD2AVI to demux the ac3s for reauthoring. If Mpeg2dec no longer needed d2v files, then I would just use DVDDecrypter to demux.

Would it create a long delay the first time a frame is accessed for the d2v to be created?

[Guest]

Originally posted by int 21h
I can only speak for myself, but I usually use Mpeg2Dec and DVD2AVI for DVD-to-DVD operations, and I just create the .d2v and use DVD2AVI to demux the ac3s for reauthoring. If Mpeg2dec no longer needed d2v files, then I would just use DVDDecrypter to demux.

Would it create a long delay the first time a frame is accessed for the d2v to be created?
If the D2V file does not already exist, yes. After the first time, the file would be detected and used.

[Cyberia]

I'm thinking of making a standalone mpeg2dec3 that can create the D2V file if it doesn't exist. Any interest in that?
YES! I believe Nic was going to do this exact thing. He may have already done some work on it or have some code/thought to share with you Neuron2.

Audio support would be greatly appreciated also!

I don't know but the changes can be easily ported to any version. If you provide me with a version that you need modified, I'll be happy to do it. First let's get the fix released and tested for 1.77.3 and then take it from there. I don't know where to find a release of DVD2AVI that supports HDTV/ATSC streams.
(ahem) read the faq: DVD2AVI FAQ (http://forum.doom9.org/showthread.php?s=&threadid=59272)

Guys, can I please make a request? There are too many versions of DVD2AVI. Could someone please simply pick the version with the highest version number and add to it any features/fixes from the earlier versions? So we have a 'Ultimate' version, and it would be the point from which all future changes be based.

The diversity of open source is good, but it is in the intrest of the community to periodically merge the codebases.

[Guest]

@hakko504

I have made a FAQ text for you, as follows:

QUESTION:
Does DVD2AVI/MPEG2DEC Lose Frames?

ANSWER (by Donald Graft):
Leaving aside the fixed versions I recently released
(see below), yes, frames are lost. This can cause serious problems
with audio sync and authoring with some tools. Not only that, but
random frame access is not handled correctly and incorrect frames
can be returned when navigating on the timeline via MPEG2DEC (and its
clones).

There are three causes for the frame loss in the faulty versions.

1. DVD2AVI fails to flush out the final frame's digit to the
D2V file before writing the 9 and closing. This causes one frame to
to be lost at the end.

2. MPEG2DEC cuts two frames from the frame count as a workaround
for 3 below. This is a kludgy hack that should not be necessary.
Thus, this together with 1 above, means
that 3 frames will always be lost. They are lost at the end.

3. If the opening GOP has B frames before the first P frame (IBBPBBP...),
then DVD2AVI generates an incorrect D2V file, in which the first
digits for the orphaned B frames are not written and all the
remaining digits are written out of place (shifted up by the number
of orphaned B frames). Also, MPEG2DEC cannot decode the B frames
prior to the first P frame, and so discards them. A number of frames
will be lost equal to the number of B frames prior to the first P
frame. They are lost at the beginning.

So, for example, if you edit a VOB and the resulting file has an
IBBPBBP... opening GOP, you will lose a total of 5 frames, with
2 lost at the start and 3 lost at the end.

In addition to the lost frames, MPEG2DEC does not implement
random frame access correctly. In fact it always throws away the first
B frames in the seeked-to GOP prior to the first P frame. If they are (say)
frames 12 and 13 (in display order) and you try to seek to 12,
MPEG2DEC will toss them and return frame 14 to you, without any warning
or indication about it.

Finally, when 3 above applies, the TFF/RFF flags in the D2V file are
misaligned to the frames.

I have created fixed versions of DVD2AVI and MPEG2DEC3 that solve these
problems. They are available here:

http://neuron2.net/fixd2v/decodefix.html

To fix these things, I started with DVD2AVI 1.77.3 and the MPEG2DEC3
from Nic's site. I rewrote the D2V file generation to be correct, even
when the first GOP is of the form IBBPBBP.... I added a pop-up message
box that appears if the first GOP is open. This warns that the first
few frames may be decoded incorrectly.

I modified MPEG2DEC to not truncate B frames prior to the first P
frame when seeking and to not unconditionally reduce the frame count by two. I
rewrote the decoding and random access code to work correctly with the
D2V files generated by the fixed DVD2AVI.

Note: Given a stream with an open first GOP, the correct frame count
is returned and seeking works, but (of course) you'll get garbage for the
orphaned B frames. That seemed the best thing to do. If the first GOP
is closed, everything is perfect.

[Guest]

Originally posted by Cyberia
(ahem) read the faqThank you. I see it now.

Guys, can I please make a request? There are too many versions of DVD2AVI. Could someone please simply pick the version with the highest version number and add to it any features/fixes from the earlier versions? So we have a 'Ultimate' version, and it would be the point from which all future changes be based.

The diversity of open source is good, but it is in the intrest of the community to periodically merge the codebases.That's a fine idea but my immediate focus is to get the decoding functionality to work properly. Your suggestion can be contemplated at a later time and in a different thread.

[hakko504]

Originally posted by neuron2
@hakko504

I have made a FAQ text for you Thank you, it has beeen added to the FAQ, along with word and PDF versions of the FAQ, downloadable from here (http://members.fortunecity.se/hakko504/dvd2avi/).

[RB]

Originally posted by neuron2
I've completed the first beta of the full fix:

http://neuron2.net/misc/DecodeFix1.0.0b1.zip

Please post your test results and/or bug reports here.

Thanks much!

OK, tested it with two R2 PAL DVDs (Meet the Parents and Starship Troopers) so far. When I load the VOBs into the new DVD2AVI, I immediately get the warning about the first GOP being open. However, the first GOP does not have "orphaned" B-Frames, it starts with "IPBB...". Does the warning really apply here? Also, I get the warning every time I try to move the slider at the bottom, effectively making it unusable ;) The warning also appears again (but only once) when I save the project or do a preview. It probably shouldn't.

Straight frameserving into CCE and the like appears to work fine via the new mpeg2dec3, but random access is definitely broken. In both MediaPlayer 6.4 and CCE, moving the slider only scans a small part of the video, seems to pick up a new small part every time you move the slider back to the start.

Again, thanks!

[hakko504]

Originally posted by Cyberia
Guys, can I please make a request? There are too many versions of DVD2AVI. Could someone please simply pick the version with the highest version number and add to it any features/fixes from the earlier versions? So we have a 'Ultimate' version, and it would be the point from which all future changes be based.

The diversity of open source is good, but it is in the intrest of the community to periodically merge the codebases. Once upon a time, I think this was one of the reasons why the SAVE-OE project was started. In any case the main versions that would need to be merged to make this 'Ultimate tool' is: Neuron2's 1.77.3 fix. This should form the base for the encoding and project creation part. Nic's (as yet) unreleased version, which includes a lot of interesting stuff, including 2-pass encoding. (Yes I've seen an alpha of this version and it looked very promising) trbarry's HDTV/ATSC/DVB capabilities from the 1.83.5 version DVD2SVCD's 1.76CLI version for the CLI options

In any case, such a project should not start until we know that Don's new fixes work correctly as this is one of the most important things that should be entered into a new version. A quick look at the SAVE-OE homepage at sourceforge.net (http://sourceforge.net/projects/save-oe/) told me that both trbarry and Nic seems to be members of the SAVE-OE project already. Maybe you could continue working from there if Don also could be added to that project. (int21h, are you listening ;))

[Guest]

@RB

Thank you for your feedback. I will correct those issues and release a new version.

[Guest]

I have fixed the DVD2AVI part:

http://neuron2.net/fixd2v/decodefix.html

It now warns only on IBBP... with closed_gop=0, and it warns only once, either on the first Preview or the first Save Project. Please test it and advise of results.

I will look at the MPEG2DEC3 part now.

[Guest]

There is still one funniness in DVD2AVI I need to fix. If you Save Project more than once, the D2V is not correct. I'll fix that in the next beta.

@RB

I cannot duplicate your random access problem with Media Player. Please describe a procedure for me to duplicate it. If necessary I will give you an FTP site to upload a VOB. Thank you.

[RB]

Originally posted by neuron2
@RB

I cannot duplicate your random access problem with Media Player. Please describe a procedure for me to duplicate it. If necessary I will give you an FTP site to upload a VOB. Thank you.
Can you test with a PAL DVD? It might be related to PAL. I'm using AVISynth 2.5.2 (September 3 CVS snapshot) and this is the AVS:

Mpeg2Source("E:\DVD2DVD-R\Meet the Parents\Source\parents.d2v")
ConvertToYUY2()
AddAudio()

Your mpeg2dec3.dll is in the AVISynth plugins directory, no other plugins there except for AddAudio.avsi. Here are the first few lines from the D2V:

DVD2AVIProjectFile
6
48 E:\DVD2DVD-R\Meet the Parents\Files\VTS_03_1.VOB
48 E:\DVD2DVD-R\Meet the Parents\Files\VTS_03_2.VOB
48 E:\DVD2DVD-R\Meet the Parents\Files\VTS_03_3.VOB
48 E:\DVD2DVD-R\Meet the Parents\Files\VTS_03_4.VOB
48 E:\DVD2DVD-R\Meet the Parents\Files\VTS_03_5.VOB
48 E:\DVD2DVD-R\Meet the Parents\Files\VTS_03_6.VOB

Stream_Type=1,0,0
iDCT_Algorithm=5 (1:MMX 2:SSEMMX 3:FPU 4:REF 5:SSE2MMX)
YUVRGB_Scale=1 (0:TVScale 1:PCScale)
Luminance_Filter=0,0 (Gamma, Offset)
Picture_Size=0,0,0,0,0,0 (ClipLeft, ClipRight, ClipTop, ClipBottom)
Field_Operation=0 (0:None 1:ForcedFILM 2:SwapOrder)
Frame_Rate=25000
Location=0,0,5,6719B

7 0 0 2 2 2 2 2 2 2 2 2 2
7 0 B3 2 2 2 2 2 2 2 2 2 2 2 2
7 0 198 2 2 2 2 2 2 2 2 2 2 2 2
7 0 27F 2 2 2 2 2 2 2 2 2 2 2 2
7 0 362 2 2 2 2 2 2 2 2 2 2 2 2
7 0 447 2 2 2 2 2 2 2 2 2 2 2 2
7 0 52E 2 2 2 2 2 2 2 2 2 2 2 2
7 0 615 2 2 2 2 2 2 2 2 2 2 2 2
7 0 6F7 2 2 2 2 2 2 2 2 2 2 2 2
7 0 7DF 2 2 2 2 2 2 2 2 2 2 2 2
7 0 8C2 2 2 2 2 2 2 2 2 2 2 2 2
7 0 9A8 2 2 2 2 2 2 2 2 2 2 2 2
7 0 A91 2 2 2 2 2 2 2 2 2 2 2 2
7 0 B8D 2 2 2 2 2 2 2 2 2 2 2 2
7 0 C71 2 2 2 2 2 2 2 2 2 2 2 2
7 0 D50 2 2 2 2 2 2 2 2 2 2 2 2
7 0 E38 2 2 2 2 2 2 2 2 2 2 2 2
7 0 F1B 2 2 2 2 2 2 2 2 2 2 2 2

I open the script in Windows MediaPlayer 6.4 and then quickly move the time line slider forward so the time display shows, say, 1:30:04. However, the sequence that plays then is clearly from the first 10 minutes of the movie. Same in CCE. If necessary, I'll try to create a small VOB to upload.

[Guest]

I fixed the problem with multiple Save Project operations. Get beta 3 from the link above.

@RB

Maybe it is having a problem when navigating to a subsequent VOB file. Please try again, but staying within the first VOB and let me know what happens. I will try this with multiple VOBs.

I don't have PAL, so if you can duplicate this with a single VOB that you can give me that would be the best.

Finally, please load the MPEG2DEC3dg.dll directly via a loadplugin command.

[Guest]

@RB

Never mind. I have duplicated the problem. Fix is in the works. Thank you for your valuable feedback.

[Guest]

@RB

Please try beta 4 from the link above. I can't test it because I have no VOBs here at work, but the problem was really obvious. It was always using the file descriptor for the first VOB. :stupid:

[RB]

Works perfectly now, thanks! :)

[Guest]

Thanks, RB! Please beat the **** out of it and report any issues if you find any, especially with the number of frames delivered and the agreement between the generated D2V file and what VOBEdit (etc.) report. In the meantime, I will start looking at save_oe, etc., and propose a plan for development of DVD2AVI/MPEG2DEC3 versions that incorporate all the desirable features and fixes.

I will also place the source code at my web site this evening.

[Guest]

Originally posted by RB
I mean, is the D2V used while decoding at all, or is it just for counting frames and locating files/offsets?It is used for counting frames, random access navigation, and for obtaining the TFF/RFF flags in display order to implement pulldown as required. DVD2AVI reorders the per-frame digits in display order when writing the D2V file.

[ssjkakaroto]

hi there neuron2 i'm getting a error with dvd2avi beta4 that didnt happen with the first version you released:

The instruction at "0x00405428" referenced memory at "0x00000000". The memory cannot be read.

This happens right at the beggining of the project creation, the d2v file it creates is completely blank.

Here are my settings:
idct: 64bit or 32bit (tested both)
field op: None
Color Sp: YUV
YUV->RGB: PC Scale
Lum. Fil. and Clip Res: Disabled
Audio Output: Disabled
SIMD: MMX, SSE MMX, SSE FPU

I'm using w2k sp4 btw

thx a lot for your hard work on this issue!

[ssjkakaroto]

Here's the D2V file that the old version created:
DVD2AVIProjectFile
1
25 D:\vobs\test\VTS_01_1.VOB

Stream_Type=1,0,0
iDCT_Algorithm=3
YUVRGB_Scale=1
Luminance=0,0
Picture_Size=0,0,0,0,0,0
Field_Operation=0
Frame_Rate=29970
Location=0,0,0,351FC

7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 82 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 10B 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 18D 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 218 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 29A 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 321 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 3A4 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 42C 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 4AF 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 53A 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 5B5 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 621 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 68B 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 6FC 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 77D 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 7E7 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 863 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 8DF 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 94C 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 9D8 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 A49 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 ABE 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 B4C 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 BE2 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 C6A 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 CF3 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 D7C 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 E0E 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 E9A 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 F2C 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 FB8 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 1041 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 10D2 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 1164 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 11EB 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 127D 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 1308 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 1395 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 142F 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 14D3 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 1567 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 15DE 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 1652 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 16C4 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 1737 0 0 0 0 0 0 0 0 0 0 0 0 0
.
.
.
.

7 0 34EF7 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 34F6F 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 34FF8 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 3508D 0 0 0 0 0 0 0 0 0 0 0 0 0
7 0 3511B 0 0 0 0 0 0 0 0 0 0 0 0 0 0 9

FINISHED

[Guest]

@ssjkakaroto

I'm going to need to get the VOB. Can you cut it to a reasonable size, leaving the beginning intact, such that it still faults? If so, I will PM you with an FTP site to give it to me.

Try using SmartRipper to cut out the first cell and see if it fails with that.

[RB]

Originally posted by neuron2
Thanks, RB! Please beat the **** out of it and report any issues if you find any, especially with the number of frames delivered and the agreement between the generated D2V file and what VOBEdit (etc.) report.
I encoded two whole movies in CCE and found no issues. Framecount is up to what IFOEdit reports, no missing frames or other visible errors. Great job, Donald! Thanks! :)

[ssjkakaroto]

ok, i'll try SR when i get home, but with dvddecrypter just the cell that is giving me problems is about 400mb long, is there any other way i can cut it? (never edited vobs before :P)

[Guest]

Cut the cell using any tool then use VobSplit (from the doom9 download page). To cut it down. Please make sure it still fails before offering it. Thank you.

EDIT: I can provide you with a mailing address if you'd like to mail a disk with the failing VOB. That might be the easiest for both of us.

[Guest]

@ssjkakaroto

Thank you for the VOB. It is clear what the problem is. I am not handling field pictures properly. It should be a relatively easy fix.

EDIT: OK, I have a fix. Just had to add a little ! character. I'll make a release tomorrow as it's past my bedtime. :rolleyes:

[ssjkakaroto]

glad i could help, thx a lot neuron2!

[Guest]

The fixed version, beta 5, is now available at my web site. It fixes field picture handling.

[Guest]

I accidentally removed some code which GKnot needs to run the MPEG2DEC3 DLL outside of Avisynth. I will restore it and release a beta 6 tonight.

EDIT: Beta 6 is now available. If you want to invoke the MPEG2DEC3dg.dll from outside Avisynth, you'll either have to rename it to MPEG2DEC3.dll or revise your DLL loading code.

[Cyberia]

Regarding P4 optimizations of the 1.77.x branch... as far as MPEG2DEC3 goes these are only going to affect MPEG2DEC3 in regards to the d2v file creation, right?

That question feels convoluted, can't think of a clear way to ask....maybe: Since you are only adding d2v file creation abilities to MPEG2DEC3, the P4 optimizations in 1.77.3 of DVD2AVI will only affect the new section of MPEG2DEC3? sheesh, that was hard. Basically, A LOT of time was spent optimizing MPEG2DEC3 for speed, and I want to make sure the changes won't affect that.

EDIT by hakko504: This better? :)

[Guest]

@Cyberia

The optimizations in MPEG2DEC3 1.10 from Nic, which I started with, are all at the lower levels -- things like the IDCT, etc. My changes are at a higher level where only existing C code is involved. So the changes I've already made cannot slow anything down. In fact,things may be a bit faster due to elimination of some copy operations in some circumstances. Regarding the contemplated addition of support for D2V file creation, that will not affect decoding speed at all and will have performance comparable to that in DVD2AVI.

I trust that this sets your mind at ease.

[Guest]

One thing I'm wondering about... Do we need to adjust the reported audio delay now as a result of these changes? I'm pretty ignorant about the audio issues. Hakko504, can you shed any light on this? Recall that in some circumstances the first two frames were dropped and with the fixes, they are now never dropped.

[hakko504]

Originally posted by neuron2
One thing I'm wondering about... Do we need to adjust the reported audio delay now as a result of these changes? Actually I don't think so. The delay is defined in getbit.c (search for AudioPTS-VideoPTS ). And as far as I can make out, the frames dropped at the start of the file are frame #2 and #3 (assuming the I-frame to be #1) but the delay is still reported compared to the I-frame! This could maybe explain some minor synch issues that have been reported earlier. 2 dropped frames would induce 66~84ms erroneous delay.
And if I'm wrong, the most likely scenario is still that the incorrect dropping of frames would adjust the videoPTS to follow the rest of the file - and the delay would still not have to be adjusted.

Disclaimer & note for neuron2- I haven't looked at the code properly in almost 6 months and I do not have the time to double check everything tonight but I think what I wrote should give you a push in the right direction. If you have any specific questions PM/email me directly (doom at hakko dot eml dot cc) and I'll go through the code a bit more thorough tomorrow.

[ssjkakaroto]

thx a lot for the fix neuron2, it's working perfectly now :)

[Guest]

Not quite perfect yet, but the old bugs in MPEG2DECx are slowly being squished. :)

I just found two more that need to be fixed. Consider this code that executes when data runs out in the first file (VOB):


void CMPEG2Decoder::Next_File()
{
if (File_Flag < File_Limit-1)
File_Flag ++;

_lseeki64(Infile[File_Flag], 0, SEEK_SET);
_read(Infile[File_Flag], Rdbfr + Read, BUFFER_SIZE - Read);
}First, suppose we have just exhausted the bits in the last and final file. We have then File_Flag = File_Limit - 1 and so we don't increment to the next file descriptor. But we do seek back to the beginning of the current file and continuing decoding it again! You might think that is OK because only the number of frames in the clip will be requested. Well, almost. But due to ISO spec, we decode one ahead for display and then should flush the last frame to the display when the bits run out. But the way this is coded, instead of properly detecting bit exhaustion and stopping decoding, it decodes the first frame of the last file again and then displays the last frame as normal (not flushing the last I or P frame). That will work OK when the first frame of the file is an I or P frame (as it often is). But if it is a B frame (due to a silly cut) then an incorrect last frame will be generated.

I have already fixed this mechanism to properly stop decoding when the bits dry up. It's available as beta 7 at my site. But there is another bug above too that I haven't fixed yet. When you go to the next file, it always seeks to byte 0 of the file. It should seek to the proper first LBA as defined in the D2V file. Again, this will cause problems when the first frame in the file is not the start of a GOP, because the D2V has discarded these, but now we will decode them due to the bug.

Finally, with 3:2 pulldown, beta 7 incorrectly truncates a frame if the last frame in the D2V has RFF set.

I will fix these two issues and release a beta 8. They are fairly minor, but since we've come this far, let's make it fully correct.

[Guest]

I've now corrected these issues and uploaded a fixed version. Please get version 1.0.0 beta 8 from my website. I am pretty happy with the correctness and robustness now, so I have placed the source code on my site as well, for those curious to see it, as well as to get a distributed code backup, an advanced source code management technique invented by the venerable trbarry. :)

I now plan to let this version soak for a while. Please give it a good workout. I will be doing some encodes using it as a normal user for a few days as well. When I'm confident that it is fully OK, I will add the same fixes to DVD2AVI's Preview, Save AVI, and VFAPI functionality. Then I will make a command line version of DVD2AVIdg.

[RB]

Thanks a lot for all the work! I'll give it a whirl...

[Zeul]

have followed this post with great interest, and after a couple of tests the output is far more accurate than ever before. excellent work and can't wait for the command line version to arrive


zeul

[NanoBot]

Hi Donald, hi anybody else,

first of all, great thanx to you, Donald, for fixing the problem with DVD2AVI und mpg2dec3.dll.

Now the question: When I am using the original DVD2AVI 1.76 and the fixed 1.77.3 from Donald with identical parameters set in DVD2AVI and the same m2v sourcefile ( I used the aurora dolby digital trailer as source), they produce different output in the d2v-file:

Original DVD2AVI 1.76:

DVD2AVIProjectFile
1
25 H:\Trailer_PAL\Aurora.m2v

Stream_Type=0,0,0
iDCT_Algorithm=2
YUVRGB_Scale=0
Luminance=128,0
Picture_Size=0,0,0,0,0,0
Field_Operation=0
Frame_Rate=25000
Location=0,0,0,2768

...

Fixed DVD2AVI 1.77.3 ( beta 8):

DVD2AVIProjectFile
1
25 H:\Trailer_PAL\Aurora.m2v

Stream_Type=0,0,0
iDCT_Algorithm=2 (1:MMX 2:SSEMMX 3:FPU 4:REF 5:SSE2MMX)
YUVRGB_Scale=0 (0:TVScale 1:PCScale)
Luminance_Filter=0,0 (Gamma, Offset)
Picture_Size=0,0,0,0,0,0 (ClipLeft, ClipRight, ClipTop, ClipBottom)
Field_Operation=0 (0:None 1:ForcedFILM 2:SwapOrder)
Frame_Rate=25000
Location=0,0,0,2768

...

Older betas from Donald will produce .d2v files with the parameter name "Luminance" instead of "Luminance_Filter", but also with the value 0,0 instead of 128,0 which is produced by the original dvd2avi 1.76

So my question is, did I found a bug ? And is this bug in 1.76 oder in 1.77.3 ?

When using those .d2v files together with avisynth.dll 2.52 and the original / the fixed mpeg2dec3.dll, both output clips have the same luminance. But when I am loading those .d2v files into "AviUtil" 0.98d, something goes wrong. The d2v files produced by DVD2AVI 1.76 are displayed with the correct luminance level by AviUtil. Older beta versions from the patched 1.77.3 are displayed to dark, and the d2v produced by the last 1.77.3 ( beta 8 ) crashes AviUtil. When changing the parameter name from "Luminance_Filter" to "Luminance" and removing the additional information in brackets, the newest d2v works with AviUtil, but the displayed clip is also to dark.

Changing the luminance value from 0,0 to 128,0 manually results in a correct luminance level displayed by AviUtil. Afaik AviUtil uses a vfapireader plugin to decode the mpg stream.


C.U. NanoBot

[Guest]

DVD2AVI 1.77.3 changed the D2V format, the name of the luminance variable, and its sense. The MPEG2DEC3dg DLL is capable of reading the new format and reversing the sense of the luminance info. In earlier betas I generated the old style D2V file, except the luminance info was still reversed. When I found out that MPEG2DEC3 could read both I just left it as 1.77.3 style.

I could change DVD2AVIdg to use the old 1.76 style D2V file. But that might break any applications that are designed for 1.77.3. I am thinking of making a command you can put in your INI file to specify 1.76-style D2V files. Will that be OK or can you think of anything better?