does dvd2avi chop off frames?

[Guest]

There's no need to be confused, because the MPEG2DEC3 and others are discarding the initial B frames in display order. You must have IBBP with closed_gop=true. MPEG2DEC3dg retains initial B frames in display order.

[jdobbs]

@neuron2

A big hats off to you. I was in the middle of a programming project in which I was creating a .D2V with my own code... I was scratching my head wondering why my frame counts didn't match DVD2AVI's. Whew -- you saved the day dude.

jdobbs

[Guest]

Here is my current thinking on the audio delay issue.

First, let's consider some specifications for audio-video synchronization. ITU-R BT.1359-1 (1998) recommended that the audio should not lead the video by more than 90ms and it should not lag by more than 185ms. The asymmetry reflects the fact that leading audio is perceived as unnatural. The ITU considers 45ms/125ms to be detectable and 90ms/185ms to be the point where the delay becomes unacceptable.

The ITU tolerances are fairly large and others are tighter. For example, ATSC recommends 15ms lead/45ms lag. DirecTV specifies 20ms/20ms but has certified boxes up to 32ms/32ms.

There is jitter due to network delays, buffering, decoding delays, etc. The instantaneous jitter can be as large as 90ms, but the time average is lower.

I cite this data to put into perspective our dilemma. We are agonizing about an offset of two frames, which is 64ms for NTSC and 80ms for PAL. The difference is within the ITU limits and, if it represents audio lag, is not even detectable according to the ITU. As an experiment, I made several encodes with different offsets differing by two frame times. I was unable to perceive any differences. In fact, we are surprisingly insensitive to simple offsets. The real sync problems are those for which the offset becomes progressively larger as the clip progresses.

Notwithstanding this, we should try to do things correctly. The problem is that it is not obvious what is correct. I have therefore made some reasonable assumptions and considered compatibility needs and have drawn a conclusion from that. Following is my analysis. I present it tentatively and welcome feedback.

The issue arises when we have an initial GOP of the form IBBP... This is very common and most of the VOBs I have ripped start that way with closed_gop=true. The two initial B pictures in display order are decodable, and they are included in the frame count in the IFO data; and because the MPEG-2 specification does not specify discarding initial B frames, we have to assume that these frames are intended to be decoded and delivered.

I have confirmed that the delay offset reported by DVD2AVI (original) and in the IFO data represents the time between the first audio packet and the first packet containing a GOP header. I confirmed that by inspecting the PTS timestamps. The question we have is whether the video (GOP) timestamp is intended to refer to the first coded picture or the first displayed picture. The MPEG-2 Systems specification specifically states that the PTS refers to the first coded access unit in the packet, i.e., the I picture. This is confirmed by noting that the PTS and DTS differ as you would expect for an I picture but not for a B picture.

Consider a VOB for which the first audio packet is at time 0 (APTS) and the first GOP is at time 83 (VPTS). If the first display picture is the I picture, and we create an AVI beginning with that picture, then we will have to start interleaving audio data beginning from time 83. Thus we need to advance the audio by 83 to achieve sync. In general:

audio advance = VPTS - APTS

If we have n number of B pictures first in display order, however, things change. Adding the B pictures delays the I picture, which means we need to reduce the audio advance by n picture times:

audio advance = VPTS - APTS - n * P

Because delay = - advance, we have:

delay = APTS - VPTS + n * P

Thus, we conclude that when the GOP is of the form IBBP, the delay must be adjusted to account for the inital B pictures in display order. This is how Release candidate 3 behaves and it seems to me that it must be viewed as correct according to the MPEG-2 specifications.

Now we are in a position to address len0x's question: Given this behavior, what are the consequences of using audio demuxed from other programs that do not allow for this adjustment with video produced by DVD2AVIdg/MPEG2DEC3dg? The audio advance will be too high by a factor of n * P and thus audio will be advanced by too much when corrected. The audio lead will be 64ms for NTSC and 80ms for PAL. Both are within the acceptable range according to the ITU. Of course, the correction factor can be adjusted before encoding to correct things.

My main purpose is to produce a decoding solution that is correct according to the MPEG-2 specifications. That is why I am not keen on dropping the initial B frames. That would cause complications in the code and defeat the purpose of the entire effort, which is to deliver all coded frames. If there are legacy issues arising from incorrect decisions by other applications, I am happy to help in working around those issues, but I am not willing to compromise correctness.

[len0x]

Originally posted by neuron2
[BIs it possible for you to just delete your posts so we can clean up this mess? [/B]

done.

lol - really bad timing from both of us ;)

[len0x]

Everything sounds fair to me.
I have one last question - what is the impact of using old d2v projects with new mpeg2dec3dg.dll ?

[Guest]

Originally posted by len0x
Everything sounds fair to me.
I have one last question - what is the impact of using old d2v projects with new mpeg2dec3dg.dll ? If the stream starts with IPBB..., the only effect is that you will lose the last frame. If the stream starts with IBBP... you will lose two frames at the start and one at the end, and if there are any RFFs they will be misaligned to the frames. The latter is usually not serious. It's just like applying pulldown.exe with a slightly different phase. :) Still, it's not a good idea to do this.

[len0x]

Originally posted by neuron2
Still, it's not a good idea to do this.

I wouldn't do that myself, just want to be able to warn others :)

Anyway, RC3 looks solid to me.

[trbarry]

My main purpose is to produce a decoding solution that is correct according to the MPEG-2 specifications. That is why I am not keen on dropping the initial B frames. That would cause complications in the code and defeat the purpose of the entire effort, which is to deliver all coded frames. If there are legacy issues arising from incorrect decisions by other applications, I am happy to help in working around those issues, but I am not willing to compromise correctness.

100% agreement here.

If the stream starts with IPBB..., the only effect is that you will lose the last frame. If the stream starts with IBBP... you will lose two frames at the start and one at the end, and if there are any RFFs they will be misaligned to the frames. The latter is usually not serious. It's just like applying pulldown.exe with a slightly different phase. Still, it's not a good idea to do this.

Donald -

Could you elaborate a bit on this? RFF's repeat fields, not frames. Does this impact using Force Film or using Telecide in any way?

Since there is no new HDTV support yet I was wondering if I dare go to the new MPEG2DEC3dg, or should wait. (or have to code something ;) )

- Tom

[Guest]

Originally posted by trbarry
Could you elaborate a bit on this? RFF's repeat fields, not frames. Does this impact using Force Film or using Telecide in any way? I assume you mean to elaborate on the misaligned flags issue. The sections that have no RFFs obviously will not suffer. The sections that are telecined from progressive frames also won't matter, because it doesn't matter with what phase you apply the telecining. There may be blips at the transitions between the sections but they will be deinterlaced if they produce combs. That's just my quick reaction; I haven't done any experiments.

Since there is no new HDTV support yet I was wondering if I dare go to the new MPEG2DEC3dg, or should wait. (or have to code something ;) ) I intended all along to add the transport stream support after the first non-beta release. Is that the same thing as HDTV support? Of course, if you'd like to take the first non-beta release code and add it and then feed back the changes to me, that would be ideal. You'd probably do it faster than me since I'd be starting from ground zero as far as understanding is concerned.

[jdobbs]

@neuron2,

I've been playing with this for a couple of days now, and I'm confused. I was hoping you could help.

As I've been writing a D2V module for a project I'm working, I've inserted a frame counter at the point that the trf values are being written (just as a check) to the .D2V file because the project requires perfect frame accuracy (it creates .AVS and .D2V files for every cell in a DVD).

I have been using LOTR, The Two Towers as a test. In the first cell I count 5929 original frames (I, P, and B) while reading. I write 5929 trf values to the file. But when I open the file with VirtualDub, or any other package, it sees 5918 frames to be encoded. That, by the way, is also exactly what I see from a D2V created by DVD2AVI v 1.77.3dg.

But I KNOW from counting that there are 5929 frames in the original -- and if I count the "0, 1, 2, and 3" values that exist in the .D2V file it also is 5929.

I seems almost as if MPEG2DEC3 is ignoring the entire last line of values written in the trf (either that or there is a coincidence that the difference between 5929 and 5918 is exactly the number of frames in that last line.

Any ideas?

jdobbs

Note added later: Please note, also that using prior versions of DVD2AVI came up with an VDub value of 5916. Also, please note that it doesn't do this on all cells. Cells 2 and 3 of that same title both have a match between the frames counted and what VDUB sees.

[FilipeAmadeuO]

Is it possible to include the option to generate a D2F file ???
Is it very handy when settings chapters (or cells) in CCE or Scenarist

[jdobbs]

My problem may be related to the "Field_Operation" flag in some way. I noticed the first 50+ frames don't have the RFF flag set at all -- while the rest of the file sets them regularly, so I set the Field_Operation to zero and changed the frame_rate from 23976 to 29970. The frame count VDUB sees in this mode matched (7397) what my code predicted -- which was actual frame_count + (rff_flag_count / 2). But since the picture_structure of the first set of frames was FRAME I didn't think it would make a difference.

Is it possible that in Field_Operation=1 mode some non-RFF'd frames are eliminated?

Added later: Does the playback in that mode simply recreate the entire MPEG2 stream and then eliminate frames, assuming some were generated from RFFs?

[jdobbs]

@FilipeAmadeuO

Was your question meant for me?

The goal of my project is to make it possible to recreate the original DVD from CCE generated output without needing to do any Scenarist reauthoring. It's a sort of "one-click" method for CCE users. My program will also take the CCE generated M2V and reintegrate it into the .VOB stream. As for CCE chapters, it would be unnecessary -- as all chapter points begin on a cell boundary anyway.

But -- I don't want to get the discussion too far off the DVD2AVI subject area... so I'll leave it there.

jdobbs

[Guest]

Originally posted by jdobbs
Is it possible that in Field_Operation=1 mode some non-RFF'd frames are eliminated? It looks like you haven't followed this thread carefully, because it is all answered right in here. The answer to your question is yes. Force film mode has to return a clip at 24fps. For sections with no RFF's, the only way to reduce their rate to 24fps is to discard frames. Some flag combinations cause frames to be added (repeated) as well, e.g., 313131...

Does the playback in that mode simply recreate the entire MPEG2 stream and then eliminate frames, assuming some were generated from RFFs? No. MPEG2DEC3 reads the flags and builds a table in memory that maps the requested frame number to the actual frame number to use. Then it decodes and delivers the frames by using the mapping. As I said, the mapping table may add or subtract frames as needed to maintain 24fps throughout.

[Guest]

Originally posted by FilipeAmadeuO
Is it possible to include the option to generate a D2F file ???
Is it very handy when settings chapters (or cells) in CCE or Scenarist I don't know what a D2F file is. Sorry.

[trbarry]

I intended all along to add the transport stream support after the first non-beta release. Is that the same thing as HDTV support? Of course, if you'd like to take the first non-beta release code and add it and then feed back the changes to me, that would be ideal. You'd probably do it faster than me since I'd be starting from ground zero as far as understanding is concerned.

Donald -

Yeah. I was afraid when I posted that you would say something like that. Downloading now. ;)

I was sort of waiting until you thought it was pretty much finished again, but that seems to be the case now. And I won't really have very much time to play with it these days but I can at least follow my own suggestion above and try dropping in GetBits.c to see if it's still compatible.

Let's take this part off line since we don't want to be running 2 different betas and I'd rather not be responsible for the current DVD2AVI/MPEG2DEC's anymore if possible.

- Tom

[Guest]

@trbarry

OK. I'm more than happy to do the work if you will help me with any questions I have.

[FilipeAmadeuO]

Oi !! The D2F is an option that the DVD2AVI 1.76+ has that generates a file that marks all chapters or cells. It is very useful for putting the chapters in CCE or Scenarist

[jdobbs]

@neuron2

Thanks. I got it working now. I appreciate your help.

jdobbs

[AndyP]

Hi

Tried using DoItFast4U with the new DVD2AVI/MPEG2DEC3 DG versions (renamed of course!) and all worked OK. But, reauthorist seems to get very confused. I noticed above that it was possible for this new DVD2AVI version to create 1.76 compatible D2V files......

My question.. is it possible to set this as a commandline switch, and can DoItFast4U be made to use this switch.. as I think this would fix the problem. I will post in the reauthorist thread also (but as its development seems dead I thought I would post here also - please feel free to correct)

Kind Regards,
Andy

[len0x]

I've just noticed that if started from CLI DVD2AVI gives you name of the project (and demuxed audio) which is totally capitalized. Is there a reason for that ? From gui it's always the same as you enter it on the save dialog...

P.S. Don, RC3 looks quite stable to me - are you planning to release it as it is or you're working on more features?

[Guest]

Originally posted by len0x
I've just noticed that if started from CLI DVD2AVI gives you name of the project (and demuxed audio) which is totally capitalized. Is there a reason for that ? From gui it's always the same as you enter it on the save dialog... Just a side effect of the parsing code. I can change it in the next release.

P.S. Don, RC3 looks quite stable to me - are you planning to release it as it is or you're working on more features? I released it as version 1.0.0. It is identical to RC3.

[len0x]

Originally posted by neuron2
Just a side effect of the parsing code. I can change it in the next release.


That'd be nice :)

My congrat on first release! :cool:

[MLS]

Andy this has nothing to do with the 1.77.3 and 1.76 d2v version differences, RA just doesnt like neurons fixed dvd2avi. I'd suggest leaving the old 1.76 and after RA scans the dif4u dir, then create your own d2vs with dvd2avidg and rename accordingly for CCE and beyond.

/MLS

[MoonWalker]

I have created a d2v file using DVD2AVIdg and I have 2 questions:

1)With DVD2AVI 1.76 I have 164662 frames and with DVD2AVIdg 1.0.0 I have 164665.Is this normal??As I read at the previous posts(as far I understand them ;) ) I must have 2 frames missing not 3.The GOP is IBBPBBP...

2)With DVD2AVI 1.76 I have the AC3 track "POLTERGEIST AC3 T01 2_0ch 192Kbps DELAY -95ms.ac3" and with DVD2AVIdg 1.0.0 I have "POLTERGEIST AC3 T01 2_0ch 192Kbps DELAY -15ms.ac3".Witch one of the two delays is the correct one.(I read the post of neuron2 about ITU standards but didn't get much :rolleyes: )...

Thanks,
MoonWalker

[Jeffster]

@ neuron2

Just a small detail, I noticed the 'About' box in DVD2AVI still says it is version RC2 :devil:

Since re-encoding that NTSC movie with success (no green frame now), I've encoded a PAL dvd, also without problems.

I haven't encounted an instance yet where DVD2AVI has altered the audio delay of the demuxed ac3 from what is reported in the ifo file, so I wonder how common it is that it does get changed.

Anyway just some feedback. :)

Jeff

[len0x]

@ All ppl who wonder about audio delay

If you didn't quite get Neuron's post about audio delay - don't worry.
The answer is simple:
1) The new delay will be quite common and it is the correct delay for using new d2v file with new mpeg2dec3dg.dll
2) if you're not using new mpeg2dec3dg.dll then there is no reason to use new DVD2AVIdg and you should stick to the old one.

Neuron2 can correct me if I'm wrong :)

@MoonWalker

2 frames are added in the beginning and 1 frame added in the end - that makes it 3 additional frames.

[hakko504]

Originally posted by len0x
@ All ppl who wonder about audio delay

If you didn't quite get Neuron's post about audio delay - don't worry.
The answer is simple:
1) The new delay will be quite common and it is the correct delay for using new d2v file with new mpeg2dec3dg.dll
2) if you're not using new mpeg2dec3dg.dll then there is no reason to use new DVD2AVIdg and you should stick to the old one.
This info condensed into Q17 of the FAQ.

[ssjkakaroto]

hi there, i always do what Q19 from the FAQ recommends (demux the audio with dvddecrypter), actually i never demuxed audio with dvd2avi :P . by doing that am i free of audio sync issues when using dvd2avidg or not?

tia

[hakko504]

Originally posted by ssjkakaroto
hi there, i always do what Q19 from the FAQ recommends (demux the audio with dvddecrypter), actually i never demuxed audio with dvd2avi :P . by doing that am i free of audio sync issues when using dvd2avidg or not? Good thinking: Why don't you try it and post the results here? :cool:

[len0x]

Originally posted by ssjkakaroto
hi there, i always do what Q19 from the FAQ recommends (demux the audio with dvddecrypter), actually i never demuxed audio with dvd2avi :P . by doing that am i free of audio sync issues when using dvd2avidg or not?


I can give you an answer right away - No, you're not. Delay in DVDderypter is the same as with older versions of DVD2AVI. So now it has to be done from within DVD2AVIdg...

I think Q19 should be updated in respect for that...

[hakko504]

Originally posted by len0x
I can give you an answer right away - No, you're not. Delay in DVDderypter is the same as with older versions of DVD2AVI. So now it has to be done from within DVD2AVIdg...

I think Q19 should be updated in respect for that... As you wish ;)

[Kermee]

Wanted to give a big thank you to DG for his hard work. It's greatly appreciated.

I just wanted to note something with FixD2V vs. the normal DVD2AVI/MPEG2DEC3. When using FixD2V, I'm having severe performance issues with FixD2V vs. using DVD2AVI 1.77.3 and MPEG2DEC3 on the magnitude of about 900%.

I'm not sure what's causing this.

It's fairly simple for me to switch back and forth since I'm using AVISynth to serve to VirtualDubMod and created two seperate D2V projects. One with the original DVD2AVI and one with DG's FixD2V. Then load the corresponding mpeg2dec3.dll/mpeg2dec3dg.dll.

Anyone else run into this problem? It's quite noticable when working with HDTV clips.

TIA.

[Guest]

I don't understand you. What is your problem and how are you making it happen?

EDIT: If you are saying that the dg versions are serving slower than the originals *when doing repeated random navigation* (such as stepping backwards), that is something that I know about, understand, and plan to address in the next version. If you are saying that serving is slower when simply playing forward linearly, then we have a new issue, but I have not noticed any such problems. So please do provide a more complete problem report.

Regarding random navigation, it is slower because for each random navigation an entire extra GOP must be decoded (to ensure that any randomly accessed frame is decodable). I have a plan to improve this in the next release.

[Stress_analyst]

I have a question regarding A/V processing methods with the new DVD2AVIdg.

A If we demux audio and video with Decrypter
Then process the video only through DVD2AVIdg (will get the same frame count as the decrypter’s demuxed video)
Then Decrypter’s audio delay should be correct (for demuxed audio by Decrypter).

B If we process the demuxed by Decrypter video through DVD2AVI (and drop a few frames from the decrypter’s demuxed video)
Then Decrypter’s audio delay will be incorrect (for demuxed audio by Decrypter).

Am I correct?

The Analyst

[rmagere]

Originally posted by Stress_analyst
If we process the demuxed by Decrypter video through DVD2AVI (and drop a few frames from the decrypter’s demuxed video)
Then Decrypter’s audio delay will be incorrect (for demuxed audio by Decrypter).

Isn't this what is happening with the current GK release? So are we likely to see some delay errors when we just install GK CodecPack 1.5 + GK RipPack 0.28.6.3?

[len0x]

actually it is exactly the other way around.

correct combinations are:
- audio: DVDDecrypter/ old DVD2AVI, video: old DVD2AVI
- audio: DVD2AVIdg, video: DVD2AVIdg

@rmagere

GK has nothing to do with that. It's just a matter where you're processing the audio (and yes, if you set up r4r to demux audio via DVDDecrypter, then with GK 0.28.6.x you have a problem)

[rmagere]

Yeah sorry I meant the Roborip - GK combination as since I update I had one movie slightly offsync not much just a tiny bit and now I know why.
Thanks.

[Stress_analyst]

@ len0x
Why will there be a problem with the following:

1 Demux Audio.ac3 and Video.m2v with DVDDecrypter
2 Video.d2v with DVD2AVIdg
3 Video.mpv with AviSynth+CCE
4 Mux Audio.ac3 (from step 1) and Video.mpv (from step 3) with ...Ifoedit

Should have better results than using DVD2AVI

Thanks

[len0x]

the rule is: if you plan processing video with DVD2AVIdg (i.e. creating d2v), then audio has to be demuxed by it as well...

[pedrop357]

If I had audio demuxed either from DVDDecrypter or DVD2AVI(old), but wanted to create a d2v with DVD2AVIdg for processing Gknot, couldn't I add (or is it subtract?) 80ms delay to my previously demuxed audio?

[len0x]

Originally posted by pedrop357
If I had audio demuxed either from DVDDecrypter or DVD2AVI(old), but wanted to create a d2v with DVD2AVIdg for processing Gknot, couldn't I add (or is it subtract?) 80ms delay to my previously demuxed audio?

DVD2AVIdg doesn't always add the delay, you have to examine the vob files to see what is the opening sequence of frames...

[bond]

just a question that came to my mind:

why doesnt anyone create a dvd2avi sourceforge project, put everything the whole 7 different dvd2avis can do into it and label it as 1.9?

this way it will be possible to make sure that the dvd2avi versions mess is one and for all cleared up, everyone who wants to distribute to it can easily join the project and do it!

yes thats the beauty of sourceforge compared to releases on private homepages, which lead to many different versions!

[trbarry]

just a question that came to my mind:

why doesnt anyone create a dvd2avi sourceforge project, put everything the whole 7 different dvd2avis can do into it and label it as 1.9?


We tried that. But it never really went anywhere.

http://sourceforge.net/projects/save-oe/

- Tom

[bond]

perhaps because its save-oe and not dvd2avi. do you understand what i mean?

a dvd2avi only project would perhaps lead to nowhere either (as not much dvd2avi development is going anymore imho) but it would help to sum up all version and will help including future additions, someone else, now unknown, will do somewhere in the future perhaps
it will just be a place where other people can go to if they want to make additions to dvd2avi (and not to another project heavily based on dvd2avi)

if it needs someone who will create such a project on sf, contact all previous authors, etc i will gladly do it if you guys will support it :)

[fritzdis]

I know this is something of an old topic, but I took a while off from encoding so I'm just reading up on this stuff.

I'm a little confused about the question of audio delay. Most particularly, I'm confused about why the delay reported by DVDDecryptor is different from that of DVD2AVIdg. I understand why DVD2AVI and DVD2AVIdg may return different delays, but I don't understand why DVDDecryptor's delay will always correspond to DVD2AVI's (old). In this thread there have been references to DVD2AVIdg "adding delay" when it had to "add frames" at the beginning. Isn't that the wrong way to look at it? As far as I can tell from this thread, DVD2AVIdg doesn't "add" frames when compared to the original .m2v. Rather, it simply avoids "removing" the frames that the old DVD2AVI removes. Assuming then that the delay from DVDDecryptor is correct for the original .m2v, shouldn't it also be correct for the .d2v produced by DVD2AVIdg?

In case I've worded any of this poorly, let me try to say it another way: The point of DVD2AVIdg (as I understand it) is that with the exception of possibly the first few frames, frames in a DVD2AVIdg .d2v should match those in the original .m2v. That is, if a scene change occurs at frame #100 in the original .m2v, then it occurs at frame #100 in the DVD2AVIdg .d2v rather than (possibly) at frame #98 in the DVD2AVI .d2v. The audio that is in sync with the .m2v would therefore be in sync with the DVD2AVIdg .d2v and not with the DVD2AVI .d2v. This seems to contradict at least part of the answer to Q19 of the DVD2AVI FAQ: Demuxing audio with DVDDecryptor and creating a .d2v with v1.76, etc. would create async files if B frames were dropped by DVD2AVI at the beginning of the video.

Another thing related to this issue that is not entirely clear to me is whether the ac3 files created by DVD2AVIdg, DVD2AVI (old), and DVDDecryptor are actually ever different. Do any of the versions of DVD2AVI actually remove or add audio? When testing on a file that seems to start with IBBP (5 frames more in the DVD2AVIdg .d2v than the DVDAVI 1.76 .d2v), the ac3 files from DVD2AVIdg, DVD2AVI 1.76, and DVDDecryptor were identical. Of course, there may be other cases (I only tested one where the frame count was at least 5 different) in which the files are different.

I understand that a lot of thought has been given to these issues, and my confusion may result from misinterpretation, but I haven't seen an explanation that makes sense to me yet.