DVD2AVI Sourceforge Project

[DDogg]

Can't we at least have resizing in dvd2avi? I have always wondered if speed and quality would be better if cropping, resizing, and some noise reduction were done at the pre-frameserve level. We have cropping now. Is it a terribly difficult job to add resizing and maybe some noise reduction?

Wouldn't this be a worthy project? Or, am I completely confused?

[trbarry]

Can't we at least have resizing in dvd2avi? I have always wondered if speed and quality would be better if cropping, resizing, and some noise reduction were done at the pre-frameserve level. We have cropping now.

DDogg -

I thought both DVD2AVI and mpeg2dec.dll already had resizing and cropping in them. I spent the weekend getting P4 versions of both programs working a bit better. I also did a really low class fix to stop things from bombing out on garbage input. I just doubled the buffer size so motion comp doesn't fall off the end as much. And I added a better (I think) aligned_malloc funtion, both for performance and so P4's don't crash.

But it looks to me like cropping is broken, at least for frame serving from DVD2AVI thru mpeg2dec.dll. It makes wierd colors. I even think I remember something about that from a complaint about 1.83.

I'll search but does anyone remember this? I'm going to try to at least get at least basic P4/SSE2 code and (non-P4) cropping, and resizing working properly. And then turn them over to Sourceforge or someone else.

But not any noise reduction code or other new features now. Sorry.

- Tom

edit: But remember resizing in dvd2avi or mpeg2dec has limited value because the IVTC/deinterlace is not that good, and you have to do those before resize. But in the cases where Force Film is good enough then integrated resize is useful.

[DDogg]

Well for me, and I think a lot of folks, FF is used 95% of the time. For some reason I had it in my head that serving a resized frame would be quicker than serving the original frame, but the more I think of it I guess you have to "reach in and grab" the full sized frame anyway so that you CAN resize it...so maybe it would not make a difference anyway.

Yeah, I can understand you not wanting to jump on that noise reduction thing :)

BTW, the last version I have looked at is 1.85. I do not believe resizing is active?

[trbarry]

I'm not sure all the various release numbers are really in order. I'm working from the version of (I think) 1.83 pre-source that went into the Sourceforge project. For frameserving mpeg2dec.dll I've got the version that was posted above, I think by Int 21h in this thread.

- Tom

[vlad59]

I just make a copy/paste of an old post I post (on page 5) :

>I just downloaded dvd2aviT2 (Trbarry's version) wich is base on >Dvd2avi 1.83 (by OGO).
>
>There was a lot of new Dvd2avi some weeks ago
>
>So to make everything Clear :
>Dvd2avi 1.84 and 1.85 new features (Author Gloval) :
>- MP3 compression with lame_enc (-> crash on my C600, don't know >why)
>
>Dvd2avi 1.76.16 new features (Author : Precis F. Neumann) :
>- ACM support for audio (-> crash on my C600, don't know why)
>- maybe something else I have no source and no readme
>
>Dvd2avi 1.76 Commandline 1.04 new features (By DVD2SVCD)
>- Commandline options
>
>And Nic additions :
>- 2-pass code and many other things
>
>I think there was no more official prog based on Dvd2avi 1.76.
>
>My main pb is v1.76.16 I can't remember where I found it.

Tom'release is based of v1.83 from OGO when he tried to implement SmartDeinterlace in it. The only thing OGO has fully implemented is the resize filters of Avisynth in DVD2avi (not in mpeg2dec.dll).

As far as I can remember, there wasn't any release of mpeg2dec.dll with crop or resize capability.


Hope this helps.

[vlad59]

For the cropping :

The crop function of DVD2AVI works fine with VFAPI but produce weird colors with Mpeg2dec.dll.

I think someone began to add crop parameters to Mpeg2dec.dll (wich is quite easy) but I don't remember if he has finished or not.

[Nic]

Ivs started to work on DVD2AVI, now that all the meddlers seem to have disappeared....

Ive had my version that Ive been using for ages (for two pass, deinterlace audio etc for ages), but im starting from scratch to make it neat & tidy :) .

The way im thinking of doing it this time is some what strange & convoluted but it seems to be pretty seemless, so I think ill stick with it

(as always im really only doing it for myself, but if I do it neatly ill release it)

The way Ive done it so far, is to add an extra save menu option which (in a simplified manner) creates a d2v, then an avs script with your set params, then uses that to frameserve the d2v in to itself(DVD2AVI)....and then compresses (1pass & 2pass) (it uses standard ICM/AVIAPI routines, but does it quicker than VDub with FastCompress)

& although that sounds really stupid :) It works great... :D
I also am going to add OGM support & put full VStrip support using the VStrip DLL in. (im not sure quite what to do for audio, ill leave that last)

Also, hopefully with Tom's support put in, SSE2 code neatly in.

Then ill be done again for a while....just thought Id let you know....ill keep you posted.

-Nic

[trbarry]

Nic -

Sounds cool. My own activities have pretty much been restricted to GetPic, allocate, and the idctmmx member. So it might pretty much be a drop-in if you wanted to add it to your more elaborate changes.

But I'm still interested in using DVD2AVI for frame serving to I'm trying to do my own changes also to mpeg2dec. (except maybe resize, if it's not there)

- Tom

[Nic]

If your looking into MPEG2DEC.DLL, any speed increase would be a major plus....Have you looked at the code for the TemporalSmoother...I know if you could add better ASM support (say SSE2) that would be very popular (as TemporalSmoothing, is so CPU intensive & used alot)

Cheers,
-Nic

ps
I havent been following the thread (as I left it when it all got silly with the talk about the name & the CVS, etc)...But I assume your latest source for those c files are in this thread? or is there a newer download? Thanks alot Tom :)

[trbarry]

Nic -

I haven't looked at TemporalSmooter. To be honest I wasn't even planning that much more work to DVD2AVI. But I do have a P4 version working. I'll probably just put the DVD2AVI source back in Sourceforge since it is based upon that version.

I also have P4 support working for mpeg2dec. But I'm not sure exactly which version it's based upon as it doesn't match the mpeg2dec stuff in Sourceforge. I'm going to try fix the crop and maybe resize and then I'll post it somewhere or replace the Sourceforge mpeg2dec stuff if that's what everyone wants.

But I don't to be in the position of first level support person fielding requests for these two, so I have mixed feelings about it all. ;)

And I'll say in advance that if VFAPI support is broke it'll stay broke. I've never used it and don't even know how it works.

- Tom

[Nic]

I can understand that :) , Thanks for the info..... :)

(The VFAPI comment made me laugh :) )

Cheers,
-Nic

[DDogg]

Tom, I am also primarily interested in the frameserving aspect of dvd2avi for svcd. If you would keep me in mind and occassionally email me a copy of what you are working on it would be appreciated. Although I am backing away from the forum a bit, I would still like to do some testing.

[DDogg]

hmmm, I found Farok's version buried in this thread that does do resizing
http://profmegot.free.fr/dvd2avi
I'll test it for speed when I get a chance. He has done some nice work on this version!

EDIT:Duh, I guess most of this stuff is being done for the divx crowd, Farok's version does not frameserve. Anybody know of a version that does frameserve with free resizing (I need 480x480 or 480x360) that I can download?

[vlad59]

Farok and OGO have modified DVD2AVI to use it as a standalone tool.

So if you want to use it as a frameserve tool, you will have to use mpeg2dec.dll.

Dividee included somme other Avisynth function in mpeg2dec.dll (temporal smoother, maybe smartDeinterlace I don't remember) but the interesting part is the D2vloader with DVD2AVI code.

[pcdvdguy]

trbarry, I've modified the "stock" DVD2AVI (1.76) to directly accept MPEG-2 transport streams. (and mpeg2dec.dll, as well)

I'd like to merge my changes into the sourceforge.net project, but I think it would be easier if you checked in your pending changes first, then I checkout files from the updated repository. I didn't add any user-interface for the user to select the stream-IDs, so I stuck these into the dvd2avi.ini file. The stream-ID is stored in the d2v project file, so mpeg2dec.dll and dvd2avi.vfp directly read the stream-ID from the project-file.

This works with the *.trp files captured on my WinTV-D, but each time I want to open a stream from a different broadcaster (who uses different PIDs), I have to quit and hand-edit the dvd2avi.ini file. Perhaps some of the more Windows-savvy programmers can update the GUI to add a dialog-box to the DVD2AVI application.

Any idea on when you might be able to commit your code changes?

[trbarry]

trbarry, I've modified the "stock" DVD2AVI (1.76) to directly accept MPEG-2 transport streams. (and mpeg2dec.dll, as well)


Yes! I've been hoping someone would do that. :)

I'd like to merge my changes into the sourceforge.net project, but I think it would be easier if you checked in your pending changes first, then I checkout files from the updated repository. I didn't add any user-interface for the user to select the stream-IDs, so I stuck these into the dvd2avi.ini file. The stream-ID is stored in the d2v project file, so mpeg2dec.dll and dvd2avi.vfp directly read the stream-ID from the project-file.

I will check them in this morning. Sorry I did not see this message last night. Have you looked at Ben Cooley's code in HDTV2mpeg2 now that displays & chooses the PID's?

- Tom (trbarry at trbarry. com)

[trbarry]

pcdvdguy -

My changes should be in CVS:

- Working P4 support, including a fix contributed to idctmmx by DmitryR for idct bug.

- New aligned_malloc function that doesn't need a fixed size table that overflows.

- Sloppy fix to double some buffer sizes so HDTV material with the usual garbage break-ups does not cause crash in motion estimation rtns.

- Something else I forgot?

If you are doing HDTV and frameserving then you might also be interested in my version of mpeg2dec.dll that also has these fixes, plus some optimizations and a fix for the chroma crop bug. I did not put it in CVS because it is not based upon the same version.

Be aware also that I cannot convince myself that the existing color conversion code is correct when doing frame-pictures (480p, 720p?) at least in mpeg2dec and maybe in DVD2AVI. I'll be looking at this some more. And is the color space the same for DVD & HDTV?

My version of mpeg2dec.dll is based upon a newer version converted by someone to .cpp (with Dividees filters) but I can't remember where I got it. If anyone remembers sending it to me or discussing it please contact me. Otherwise if there are no objections I may just blot the one in CVS with this one. I just have a total brain outage on the source of this.

Any comments or objections? Anybody?

- Tom

[pcdvdguy]

> Have you looked at Ben Cooley's code in HDTV2mpeg2 now that displays &
> chooses the PID's?

Yes...Ben Cooley's program was my inspiration for making these changes to DVD2AVI!

As I noted earlier, the my main obstacle to adding native program-selection was the Windows-programming aspect (GUI.) So for now, I've left the PID-selection in the dvd2avi.ini file. This is definitely not a good long-term solution, and it's only temporary until some Windows coders can clean this up and add a nice dialog-box for selecting program-IDs.

> And is the color space the same for DVD & HDTV?

The color-space is specified somewhere in the Video_seqence_header. I'm pretty sure the default color-space is CCIR-601 compliant (which means a pixel-magnitude scale of 16-240. In most video broadcast applications, I think the values 0-15, and 241-255 are reserved for other data. But the guys at avsforum.com would know more about this than me!)

How should I go about adding my changes to the CVS repository? I only have a Celeron2, so once I make my modifications, I can't re-verify your code changes. (In other words, I might break some of your code, but not even know it, because I don't have SSE2.)

** UPDATE **

hmm, I can't seem to find the dvd2avi.vfp (VFAPI plugin filter) in the CVS repository. I followed the directions (for WinCVS) posted a few pages back. What am I missing?

[trbarry]

pcdvdguy -

Let's worry about Gui changes later. That's not my own strong suit either but I'll probably eventually add that if no one else does. Could you at least make it so if the ini file specifies the PID's as zero then it will take the first (or lowest?) pair it finds? I think that's usually correct for most HDTV stations.

On the color issue, the default for HDTV is BT.709 unless specified in that sequence_display_extension. I sampled about 3 HD clips from different sources by setting stops in DVD2AVI (or mpeg2dec?) and none of those headers were present. That is probably good because the data would have then been ignored anyway.

I don't know if specifying PC_Scale (full luma range?) is enough to handle this right for HDTV clips but I do notice that some dark HDTV scenes seem awfully muddy for some reason, even before re-encoding. But my UPN Buffy up-converts are always dark anyway, so who knows?

I guess you should just put your code back in CVS when done. That makes it easiest to note the changes. It's not likely you will break any P4/SSE2 code unless you are modifying those sse2 specific assembler sections. The only exception is making sure to use aligned_malloc to keep the data buffers on at least 16 byte boundaries.

I'm looking forward to trying this. :)

- Tom

[trbarry]

** UPDATE **

hmm, I can't seem to find the dvd2avi.vfp (VFAPI plugin filter) in the CVS repository. I followed the directions (for WinCVS) posted a few pages back. What am I missing?


I have never used vfapi and don't know how it is created. You can get one in the Gordian Knot package here but I don't know what source you would use and it likely wouldn't help with your ATSC streams.

DVD2AVI can encode files directly by saying 'save avi'. Or if you 'save project' then the actual file reading and mpeg2 decoding happens in mpeg2dec.dll instead, passing the decoded data off to Avisynth or whatever. I think vfapi is similar but I never built one.

If there is not a vfapi project in sourceforge then check out www.trbarry.com/DVD2AVIT3.zip . This has the current source I was working on and should have the vfapi project as of the 1.83 pre-release that I started from. But I've made no attempt to change it and have no idea if it works.

- Tom

[pcdvdguy]

WinCVS tells me I don't have write-access to the forum. Looks like I forgot to email the admins and ask for write access :)

trbarry, here are my code-changes,

http://members.tripod.com/~liaor/dvd2avi_cvs.zip (for DVD2AVI.EXE)
http://members.tripod.com/~liaor/mpeg2dec_cvs.zip (for mpeg2dec.dll)

for DVD2AVI, you should be able to unzip my changes into your source directory (overwrite older files.) Then recompile!

the mpeg2dec.dll is based on the 08/16/2001 release, from the archive at http://www.davetech.org/software2.htm

about the vfapi module...
I thought it was called whenever you read a *.d2v file from another application which supports VFAPI (TMPEG, BBMPEG, etc.)

> On the color issue,

Ok, I have to admit I was talking out of my rear :) I assumed DVD MPEG-2s and HDTV MPEG-2s used the same color-space, but it's been so long since I've looked at the innards of an MPEG-decoder, I no longer remember what I am talking about!

> Could you at least make it so if the ini file specifies the PID's as zero then it will take the first (or lowest?) pair it finds?

Unfortunately, this requires the program to analyze the stream for several megabytes. I like your idea, but I will defer implementing it until we have a nice dialog-box which lists the located programs. (I look at the first 2048 bytes, to determine whether the input-file is an MPEG-2 transport stream.) For now, you'll have to hand-edit the dvd2avi.ini file and enter the (hex) VideoID and AudioID streams. If you want to add this function yourself, a good place to put it would be in mpeg2dec.c.

Oh, if replacing an older dvd2avi.exe in the same directory, be sure to delete the dvd2avi.ini file the first time you run! The new dvd2avi needs to create an extra entry in the DVD2AVI.ini file, and this only happens if it can't find the file.

[Nic]

Wow, it seems like many people are working on DVD2AVI,

If you want to specifiy exactly how you would like the dialog box for pid selection would like then tell me & ill whip it up :)

Cheers,
-Nic

[ShinerSpoon]

I have always wondered what exactly happens when a D2V is opened in TMPG or avisyth via mpeg2dec in an avs file. Could one of you consider explaining the actual process that delivers frames to an encoding application using these two methods?

I know this is a big request, but I am sure it would be appreciated by many.

[trbarry]

pcdvdguy -

I've merged the dvd2avi codes and it seems to work. Kudos! ;) ;)

(what does "props" mean that everybody says these days?)

CVS is updated. I added your newer readme file as a text file since CVS doesn't like doc.

I'm still looking at the mpeg2dec code. I'll have to add the aligned malloc's & buffer size stuff or else my P4 code won't work anymore.

I still haven't figured out what version of mpeg2dec.dll I'm workng with but it seems to be newer than others. It has dividee's filters and is converted to cpp.

I guess I'll just post it all if no one says anything. And then I guess merge it into the save-oe project somewhere later. My apologies in advance to whoever did the newer updates in it and deserves credit.

- Tom

[trbarry]

I have always wondered what exactly happens when a D2V is opened in TMPG or avisyth via mpeg2dec in an avs file. Could one of you consider explaining the actual process that delivers frames to an encoding application using these two methods?

ShinerSpoon -

Let's assume you are using DVD2AVI to make a project file and then frameserving using MPEG2DEC.dll/Avisynth/Virtualdub to encode.

During the first step DVD2AVI has its own version of mpeg2dec.dll code imbedded in it which it uses to decode for preview, or if you save the avi file directly.

But if DVD2AVI saves a project file it is not actually taking the time to decode the entire file. It just spins thru it and notes the offsets of of each Group of Pictures that can later be self sufficiently decoded. It saves a list of these offsets, along with the offset of each frame, in a text file of yourprojectname.d2v, along with some other parms and preferences that you may have chosen. You can browse that file with any editor but you have to know what you are doing if you change anything.

Now say we open Virtualdub, and within that we open a file of type .avs that happens to be an Avisynth script.

When Avisynth was first installed it registered the Avisynth.dll file to get control whenever a .avs file is opened. This acts like a system hook on the file system.

So vdub opens the file and Avisynth gets control. It reads the avs script and builds a little program in memory.

Avisynth evaluates the avs script once expecting the result will be a record to return. But the beginning of the avs script would have had a command at the top like:

LoadPlugin("D:\mpeg2decB\debug\MPEG2DEC.dll")

This would have told Avisynth to load the mpeg2dec.dll which has yet another copy of the mpeg2 decoder in it. But this version does not read an mpeg2 file sequentially.

The avs script would also had a line in it something like:

videoclip=mpeg2source("D:\D_VCR\Buffy\M2v\buffy_flood_30.d2v")

When Avisynth executes the above script line the first time (at open) it calls the loaded mpeg2dec.dll module and passes it the file name of the d2v project file you specified when you did a "Save Project" in DVD2AVI.

Mpeg2dec reads that .d2v project file and from it gets parms, the real target mpeg2 file name(s), and the table of file offets mentioned above. It then returns some of that needed information to Avisynth.

Avisynth then passes back control to Virtualdub with all the needed info as if a real file was opened. But the attributes returned reflect also the effects of the rest of the avs script. This is done by magic.

But now vdub goes to read a record of some specified number. This causes Avisynth to be invoked again, with a read request for a certain record number. Avisynth calculates the return value of the script. But that (usually last) line of the script in turn relies on a previous command which eventually relies on a record from videoclip.

And Avisynth recognized that videoclip variable as being something that requires calling mpeg2dec.dll to get a record.

Avisynth calls mpegdec.dll for the needed record (not always the same number as vdub wants), grinds it through all the avs functions and returns the result to virtualdub which thinks it has read a record. And around it goes.

Whew! Don't I blather! But you asked. ;)

- Tom

[trbarry]

Wow, it seems like many people are working on DVD2AVI,

If you want to specifiy exactly how you would like the dialog box for pid selection would like then tell me & ill whip it up


Nic -

Nifty. I'll let pcdvdguy reply but probably it would be just like the audio tracks are selected now, except pcdvdguy's code could maybe supply the proper pid numbers. You could select both a video and an audio pid.

And usually the network program is using the lowest pid numbers for their main program. I think they have found that some set top boxes have trouble with other sub channels. To be honest it is probably just 11 and 14 on most channels. But there are exceptions.

I wonder how different DVB is?

- Tom

[pcdvdguy]

trbarry, I don't have an HDTV in my house, just a WinTV-D card (for my PC), so a lot of 'systems issues' escape me. I'm not trying to duck out of responsibility here, but I think you're a better candidate to direct the GUI-portion of the transport-stream code. You have a better understanding of the programID organization, and access to more test material.

> except pcdvdguy's code could maybe supply the proper pid numbers. You could select both a video and an audio pid.

Currently, the code in getbit.c isn't really setup to 'scan' for pids. In my brief look at dvd2avi's code, I believe the best place for this is mpeg2dec.c. If I'm not mistaken, that's where the application opens the input-file.

> I still haven't figured out what version of mpeg2dec.dll I'm workng with

It's not the same one at http://www.davetech.org/software2.htm ?
That package's source code seems to be only *.cpp, and it's the one I used.

...

On another note, when I open 1920x1080i MPEG2s, my DVD2avi window gets cropped on my desktop. I normally set my desktop resolution to 1152x864 or 1280x1024. I made an experimental change to gui.cpp, which automatically halves the main-window's size (if hsize > 720 or vsize > 576.) This works, but the preview window (Directdraw overlay) is still setup for 1.0x zoom, so only the video's upper left quarter is visible! Therefore, this modification isn't part of the code published on my website.



// from gui.cpp

void ResizeWindow(int width, int height)
{
// rsl
if ( Coded_Picture_Width > 720 || Coded_Picture_Height > 576 )
{
// Cut preview-window down by HALF (in both X and Y directions)
width = width >> 1;
height = height >> 1;
}
// we've cut the application window size in half, but the directdraw overlay
// surface is still configured to render at 1.0x! This means the preview
// window will 'crop' the disaplyed-video to the top-left corner!

MoveWindow(hTrack, 0, height, width-4*TRACK_HEIGHT, TRACK_HEIGHT, true);
MoveWindow(hLeftButton, width-4*TRACK_HEIGHT, height, TRACK_HEIGHT, TRACK_HEIGHT, true);
MoveWindow(hLeftArrow, width-3*TRACK_HEIGHT, height, TRACK_HEIGHT, TRACK_HEIGHT, true);
MoveWindow(hRightArrow, width-2*TRACK_HEIGHT, height, TRACK_HEIGHT, TRACK_HEIGHT, true);
MoveWindow(hRightButton, width-TRACK_HEIGHT, height, TRACK_HEIGHT, TRACK_HEIGHT, true);

GetWindowRect(hWnd, &wrect);
GetClientRect(hWnd, &crect);
Edge_Width = wrect.right - wrect.left - crect.right;
Edge_Height = wrect.bottom - wrect.top - crect.bottom;

MoveWindow(hWnd, wrect.left, wrect.top, width+Edge_Width, height+Edge_Height+TRACK_HEIGHT, true);
}

[trbarry]

I have an HD RPTV but my only receivers are a WinTV-HD, HiPix, and Wintv-d cards. Only the WinTV-HD is currently installed. The HiPix is new and I haven't tried it yet.

I'll put in a pid scan if I can steal it from Ben's code and Nic can do the gui.

When opening a large image like 1080i I have to immediately go into crop/resize and click reset. This has been true since 1.81 or somewhere. But then of couse I can't see it all so I crop and resize.

If I am frameserving through the new mpeg2dec.dll then the crop will be honored (now works) and the resize is ignored so it works out well.

I've merged the two mpeg2dec's now and am testing. News soon.

- Tom

[ShinerSpoon]

Tom, it was real decent of you to take the time to explain all that. While I have used the functionality many many times I never quite 'got it' as far as what was really going on. It worked and I was happy. Now I better see how the process flow is.

On that note, I am confused on why the resize works so well when saving an avi, but not via mpeg2dec or opening the D2V in TMPG. I would assume from what you said that the internal mpeg2dec must have more funtionality than the external one? Or, is it something to do with the actual frameserving process?

[trbarry]

ShinerSpoon -

Glad to help.

The main reason that mpeg2dec.dll doesn't resize yet is just because there is no code in it to do that. I've thought about it a couple time but I use it mostly with Avisynth and there wouldn't be that much of an improvement in doing it in mpeg2dec vs. just adding a resize filter to my script.

And doing it in mpeg2dec has the disadvantage that you can't then do a deinterlace or IVTC filter first so it may be some time before I add that, if ever.

I was mostly just concerned with speeding it up and it seemed a shame that the cropping was busted since the code was already in.

- Tom

[trbarry]

@pcdvdguy, @Nic, @* -

I have pcdvdguys changes integrated into my own and I put the whole thing including source & dll at www.trbarry.com/mpeg2dec2.zip

This is just a TEST version!

It handles regular vobs plus ATSC streams with the limitations pcdvdguy mentioned. It also can use DmitryR's P4/SSE2 code when available and when iDCT_Algorithm=5 is specified the .d2v file (from the newer DVD2AVI, or edit).

It handles cropping without creating strange colors but will still ignore resize parms.

It also doesn't crash as easy on HDTV or DVB garbage data.

It should be a bit faster on P4's and I'm still looking for other places to optimize it.

As far as I can tell it's based upon the source from dividee's site or davetech but I'm still pretty sure I was given a copy that already had a start on the P4 code. But I guess it doesn't matter.

Just copy mpeg2dec2.dll out of the zip and call that instead of your old one. There's a readme file in the zip.

- Tom

[A_Pleite]

@trbarry
Quite interesting, your explanation
When I read this, I thought that the d2v-file is only really necessary to manage b-frames, right :confused:

Question:
If I encode a (mpeg2-)avs with VDub, does the mpg2dll decompress the frames in an intelligent order (there are problems with b-frames in divx5)

Is there a binary of the new DVD2AVI somewhere (I´d like to test it)
If you don´t know, please mail: vip_nr9582@yahoo.de

[trbarry]

When I read this, I thought that the d2v-file is only really necessary to manage b-frames, right

I think there a number of reasons why it's needed, but mainly.

- If you just open up an mpeg2 file with a dshow filter or by reading the beginning of it you don't know exactly how long it is, or how many frames. But programs like Virtualdub need this info.

- For Virtualdub editing or even for some filters (say Decomb) it is necessary to be able to read records randomly, by record number.

DVD2AVI makes this possible by storing the needed info in the .d2v file.

If I encode a (mpeg2-)avs with VDub, does the mpg2dll decompress the frames in an intelligent order (there are problems with b-frames in divx5)

B-frames have been a part of mpeg2 since the beginning. I'm not aware of any problems there but I didn't go looking in the code for them.

Is there a binary of the new DVD2AVI somewhere (I´d like to test it)

I've added some other P4/sse2 optimizations that I picked up from various places, to both mpeg2dec and dvd2avi. I'll try to post test versions of both later today. I'll report back here when I do.

- Tom

[A_Pleite]

Thanks for the post :)

but
Originally posted by trbarry

...
B-frames have been a part of mpeg2 since the beginning. I'm not aware of any problems there but I didn't go looking in the code for them.
...


I think the Question was taken too unclearly. I meant: You can´t deencode the mpeg2-stream in the same order as the encoding. Does the encoder (VDub) ask for each frame seperatey or does avisynth (or mpeg2dll or VDub) notice that more than one frame is needed in a queue? If the encoder asks for each frame seperately, the decoding should be slow (so maybe this question is a bit stupid, since they are a lot of intelligent guys who spent all their speartime for speeding up this kind of tools and dlls) :)


I've added some other P4/sse2 optimizations that I picked up from various places, to both mpeg2dec and dvd2avi. I'll try to post test versions of both later today. I'll report back here when I do.

"MI" is not in Europe, right ;)

A_Pleite

[trbarry]

A_Pleite -

I think the Vdub problem was in trying to accept an encoded stream with b-frames out of order. In the case of mpeg2dec the frames are decoded and in order as Vdub gets them. I prepared a longer reply but my browser hiccupped when I hit enter and it's gone. :(

I've put TEST versions of the two projects including source and bin at

www.trbarry.com/DVD2AVIT3.zip and
www.trbarry.com/mpeg2dec2.zip

These have the P4 optimizations and other changes mentioned above. Just copy out the renamed dvd2avi2.exe and mpeg2dec2.dll modules.

- Tom

[int 21h]

Hmmm, this seems to go slower than the regular mpeg2dec build...

Compiling via Intel Compiler now.

[trbarry]

Strange, it was faster for awhile.

Maybe I left some debugging stuff in it. I'll check.

- Tom

[macdaddy]

The mpeg dll speeds things up for me, and I tested serving over a network. I didn't think I would get a boost at all because I am using amd hardware. The speed bump isn't huge (and I was serving over a network) but every little bit helps. Thanks for sharing...

Ditto with the new dvd2avi exe (again over a network). Everything seems to work. btw, what is a session, and why would I want to save or load one? I have never seen that before (and I've been using the program for years)...
___

I assume that is the same pcdvdguy from avs. The cross-pollination is pretty sweet...

Thanks again.

[trbarry]

Hmmm, this seems to go slower than the regular mpeg2dec build...

int 21h -

If you are testing on a P4, make sure that the idct type in the d2v file=5. Otherwise it will not use the P4 version of the idct code. You can do this by using the new dvd2avi and selecting sse2 idct or you can just edit the d2v file.

But I admit it is not as fast as I thought. If I one at a time remove my performance improvements, recompile, and test, they each seem to have made a difference. But so far the net gain is not really enough to beat the version I can get with Gordian Knot. Though it's mostly too close to call.

So there is either some better code with that Gordian Knot version or someone knows a bit more about C++ compiler settings. This would not be a surprise to me so if anyone is conversant in VS6/C++ build options I'd love to have some feedback. The project and workspace files are in the zip so maybe it will be obvious to someone.

But I guess it all still gives me HDTV/ATSC streams & cropping thrown in for free, so what the heck.

And I'm not done yet.

- Tom

[ShinerSpoon]

Tom, just in case you overlooked this:

http://aquaplaning.20m.com/

I think that was the source Dividee modified. The aquaplaning source was a mod of the original source from Mathias Born.

[int 21h]

Originally posted by trbarry


int 21h -

If you are testing on a P4, make sure that the idct type in the d2v file=5. Otherwise it will not use the P4 version of the idct code. You can do this by using the new dvd2avi and selecting sse2 idct or you can just edit the d2v file.

But I admit it is not as fast as I thought. If I one at a time remove my performance improvements, recompile, and test, they each seem to have made a difference. But so far the net gain is not really enough to beat the version I can get with Gordian Knot. Though it's mostly too close to call.

So there is either some better code with that Gordian Knot version or someone knows a bit more about C++ compiler settings. This would not be a surprise to me so if anyone is conversant in VS6/C++ build options I'd love to have some feedback. The project and workspace files are in the zip so maybe it will be obvious to someone.

But I guess it all still gives me HDTV/ATSC streams & cropping thrown in for free, so what the heck.

And I'm not done yet.

- Tom

I am using a P4, and I did edit the .d2v file :) It still however, does not beat the version I made (from the original source code), with the Intel Compiler (P4 specific generated code, specified with switches to the compiler).

I also can't get your source to compile with the Intel Compiler because of the macros defined on the commandline (Mpeg2dec), if I remove those, it compiles, but then doesn't link, I get unresolved externals, I really didn't look at it too much, so its probably an easy error to fix.

Honestly, I think the original SSE MMX iDCT is as-fast as the new SSE2 version.

Its getting closer to the point where there is not much left to speed up, except by switching to more inline asm, unrolling loops (which the Intel compiler already does), vectorizing loops (which the Intel compiler does), etc.

[trbarry]

Honestly, I think the original SSE MMX iDCT is as-fast as the new SSE2 version.

Its getting closer to the point where there is not much left to speed up, except by switching to more inline asm, unrolling loops (which the Intel compiler already does), vectorizing loops (which the Intel compiler does), etc.

I posted elsewhere about this a couple months ago but I think there was about a 5% speedup in dvd2avi with the sse2 idct. And at the time I was using a trial version of Intel Vtune which said the lion's share of dvd2avi processing was in the various form_prediction functions so I was glad to get sse2 code for that. But I admit it doesn't seem to have helped much in mpeg2dec.dll yet.

I don't have the Intel compiler but I'm going to try VS6 compiles of some of the other mpeg2dec sources I have. If I start from the best one and retrofit the other changes into it then I can compare better.

But I'm a little confused at this point. The changes helped for dvd2avi and it does substantially all the same things.

But I'm first going to see if there was a penalty for the C++ conversion of mpeg2dec. I don't always trust C++ to not be doing something helpful (but slower) behind the scenes.

But I'm not trying to open a debate about the merits of the languages here, just performance issues.

- Tom

[Dr. Frankenstein]

Hi Tom,

I just removed the slash in front of MPEG2DEC_EXPORTS, so I wrote /D "MPEG2DEC_EXPORTS" instead of your /D /"MPEG2DEC_EXPORTS". After this I could compile using Intel Compiler and I doubt that MSVC would compile that (but I haven't tried).

Int 21h, you can always try to use MSVC and only use Intel Compiler on the files that you really care about by setting _USE_INTEL_COMPILER settings for the file.

Dr. Frankenstein

[A_Pleite]

I made a short test with ssemmx, saving an avi directly (I know that this hasn´t really been optimized, but I wanted to see it ;)). I used the xvid-null->test speed option.
The new version of DVD2AVI is ~2% faster than the old one ;)

[trbarry]

int 21h -

I did a bit more thorough testing which leads me to believe your Intel compiler is just more efficient. I downloaded the current Doom9 mpeg2dec source (including binary dll) again and compared times.

I made a short d2v project with the old dvd2avi 1.76, a clip that takes about 4 minutes for xvid 1st pass using Avisynth and Vdub.

I then tried it with the mpeg2dec binary from Doom9. 3:56 minutes

I compiled the Doom9 mpeg2dec source and tried that. 3:56 minutes

I tried it with my sse2 version, but still with idct=2. 3:45 minutes

I then edited the d2v file so idct=5. 3:38 minutes

Finally I redid the d2v file with the new dvd2avi and specified that mpeg2dec should do the cropping, removing the crop filter from my avs script. 3:31 minutes.

I did each test twice. The results varied by maybe a second and I took the average, summarized below with time in seconds, and savings:



Test Seconds Tot Pcnt Saved

mpeg2dec binary from Doom9 234

Doom9 mpeg2dec source 234 0.0 %

SSE2,IDCT=2. 225 3.8 %

SSE2,IDCT=5 218 6.8 %

SSE2, IDCT=5, mpeg2dec crop 211 9.8 %


The last 3-4 % should apply to anyone, even without sse/sse2.

If the problem is with my compiler settings or libraries at least they are no worse than that used on the Doom9 binary.

My previous experiments with DVD2AVI and Vtune suggested about half of the cpu time was spent in DVD2AVI with the rest in xvid, testing with direct save. If this is true and also applies roughly to mpeg2dec, then this means on P4's the new mpeg2dec is almost 20% faster even though the total job is only 9.8% faster.

I'm hoping if you compile this with your Intel compiler you will see the same proportions, and for a greater total savings. If so, can I have a copy of the binary?

Is the Intel compiler expensive?

- Tom

[int 21h]

@Dr. Frankenstein:
Thanks for the tip, I'll try it when I get home.

@trbarry:
I'll do a more complete test when I get home tonight, and I should be able to compile with the Intel compiler due to the Doctor's tip for the compiler. I'll be glad to give you a binary of the compiled .dll. I usually just pass the compiler all of the Pentium 4 specific options, and cross my fingers and hope it optimizes and vectorizes stuff for me, because I'm usually not intelligent enough or blessed with enough time to do it all manually (vectorization, loop unrolling, etc).

I'm not sure how much the Intel compiler is because I have access to it through my employer.

The only other thing I can think of is a problem with my OS installation. It seems to me, I used to be able to get about 1.4 with CCE, and now I get about 1.026 (After a reinstall of Windows a month or so ago), so I think I may have installed drivers for the system in the wrong order and maybe something isn't working right..

[ShinerSpoon]

Tom and all, I completely realize this is a work in progress and a test version only, but I was curious if the besweet option in the version you linked is operational (since it was not greyed out)? If so, would anybody know the proper setup to use it or where to put the besweet files (or dll?)?

[int 21h]

Originally posted by ShinerSpoon
Tom and all, I completely realize this is a work in progress and a test version only, but I was curious if the besweet option in the version you linked is operational (since it was not greyed out)? If so, would anybody know the proper setup to use it or where to put the besweet files (or dll?)?

It isn't operational, I haven't had nearly enough time to work on it..maybe someday I'll find some free time. (I have the task of BeSweet Integration)

[Nic]

Not to step on anyones feet or anything...but I have been discussing BeSweet with DSPGuru over past few days & I should have it implemented by tomorrow or the next day.

I finished my initial VStrip integration, its very basic. But does the job for now.

-Nic

[A_Pleite]

My first DVD-Backup(2001) with the new version (remember, ~2% faster) is now almost ready. :D

@nic an included vstrip.dll means that we don´t have to store the whole unencrypted DVD on our HD´s anymore, right? ;)