DVD2AVI Sourceforge Project

[trbarry]

I don't have any strong opinions on the name, or what gets deleted. SAVE or save-oe is fine with me.

But I'd like to be able to start using CVS. I started to take a look at the Sourceforge Setup Doc (https://sourceforge.net/docman/display_doc.php?docid=774&group_id=1) but it seems to require the adminstrator be running Unix and be knowledgable in it, which sadly I am not.

Has anyone successfully used CVS with this yet? We seem to be sort of stuck. There is probably some simple step the admin or user has to do yet.

Can anyone who knows anything about this sort of browse through the save-oe project and see if there is something obvious? Do we even have a source tree?

- Tom

[int 21h]

If someone can add me as a developer to the sourceforge project I can see what I can do.

I may just start a derivative tree anyways, because honestly I won't have any use of many of the changes in the DVD2AVI interface itself. (i.e. things that don't port over to mpeg2dec.dll)

DSPGuru is helping alot, and soon I should have a DVD2AVI that processes SVCD audio from start to finish when you save your project.

[Julien]

The memberlist :
https://sourceforge.net/project/memberlist.php?group_id=44406

Tomorrow my friend will explain to me how does it work (for french people please come here (http://www.media-video.com/forum/viewtopic.php?t=949) (friday 20/21h GMT+1) to read his explanation and, please, try to translate it in english to this forum !)

Thanks !

[int 21h]

You can all praise me later.

I fixxed the goofed up release thing, and hid it. I also commited the save-src.zip into the CVS.

TADA

I can also probably write up a short thing on how to do this for Win2k if anyone needs help.

You can check out the code here: http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/save-oe

[trbarry]

int 21h -

I guess I'll praise you now because we might be busy later. ;)

Thank you. Good work.

This might start things moving along a bit.

Do your audio plans include handling/keeping ac3? That would certainly be nice.

- Tom

[int 21h]

I was planning on changing the code so that instead of doing the ac3dec thing that it does now, it just called on azid.dll to decode the ac3 frame, in any event, the current ac3 code will stay intact, and my ideas were going to be added as an alternative. I know that Jackei spend a long time perfecting and fine tuning the AC3 code according to spec, and I wouldn't want to disrespect him in that reguard.

[int 21h]

In other news, DVD2AVI doesn't compile under the Intel compiler.. some type of error in memcopy with prefetch asm commands.

And are we sure to what extent or what reason there are errors in the SSE2 code?

[trbarry]

The only SSE2 error that I'm aware of is a case, according to DmitryR, where aligned_malloc will not align properly on a 16 byte boundary if it goes into overflow storage. Dmitry suggested just doing a

#define POOL_SIZE 8192 (or larger)

until we find what is likely a memory leak.

And regarding the ac3. Are you saying there is already a way to pass the ac3 audio along with the video to Divx4 and not have to put it in with a 2nd pass? I don't want to decode it, I want to keep all the nice sound of my HDTV captures, and it is a relatively smaller percent of the space on HDTV. But it is a pain putting it back in with Vdub_ac3 later. The delays don't seem to come out right and I am really not very good at doing it manually. Is there a better way?

- Tom

[int 21h]

No, but adding the code to interleave the AC3 into the avi should be rather trivial, just a small cut/paste and mod from vdub ac3.

[trbarry]

Yes! Now that would be nifty. And I suspect some folks over on AV Science forum would be grateful too.

- Tom

[int 21h]

Was reading a bit tonight in one of my favorite books entitled "Intel(r) Pentium(r) 4 and Intel(r) Xeon(tm) Processor Optimization". On page 3-4, there is an interesting section entitled: Checking for Streaming SIMD Extensions 2 Support. That is followed by some abstract explanations and some code listings. The code that is presented is much much easier and less complicated than the current code in use (AMD stuff), and it will plug right in to the existing CPU detection code.

[trbarry]

I thought checking for SSE2 is just one more bit flag in the word returned from cpuid. I put that into the one already in CVS, along with a menu item for the new IDCT.

The complicated cpu checking code was in the old AMD samples where you didn't even know if the cpuid instruction was supported but I think we now assume at least that much, with maybe apologies to the 80486 users.

But if you have those Intel manuals now, can you figure what the cache line size is. I've seen it both as 64 and 128 in different Intel manuals for the P4.

And BTW, I indeed can check out stuff from CVS now. Haven't put anything back in yet.

- Tom

[Julien]

Thanks a lot int 21h !! :D

Here is a code optimiser : Quexal (http://www.tommesani.com/) (benchmarks (http://space.virgilio.it/insignis@tin.it/tommesani/MMXPrimer.html#ExamplesBenchmarks))

Is it interesting for us ?

[Slyde]

Many thanks int21h, serious things can start now ;)

We should list current improvements and TODO list.

Currently I think these features can be quickly added to DVD2AVI :

- Automated 2 pass encoding
- SSE2 support
- Command line parsing
- Audio compression with ACM or Lame
- Deinterlacing

Most of these are already implemented in custom builds of DVD2AVI, so merging them to the release on the CVS should be quickly done.

TODO list :

- Extensive command line support (lot of switch like MPEG2AVI for example to stop using project files) (maybe DVD2SVCD's release improvements would fit)
- Bressenham resize : slightly faster than Bilinear and is for many people as good as Bilinear.
- IVTC
- Noise filter (AVIsynth one should fit)
- Frame-precise Range selection (it's not very precise currently).
- .ifo file parsing to display audio stream langage instead of stream number. Prevent extracting the wrong stream ;)

We should also decide what will SAVE become :

- a more powerfull video/audio encoder ?
- a complete solution like GordianKnot ?
- a tool which doesn't need VirtualDub to produce usable AVIs ?

[Julien]

Originally posted by Slyde We should also decide what will SAVE become :

- a more powerfull video/audio encoder ?
- a complete solution like GordianKnot ?
- a tool which doesn't need VirtualDub to produce usable AVIs ?
The most optimised frameserver :D

[Slyde]

It is already more than a frameserver as you won't need VirtualDub anymore with all features I listed implemented. I think it should become for AVI users the fastest standalone AVI encoder as :

- It does resize/cropping.
- It will for sure do audio encoding/AC3 mux.
- It does frame range selection.
- It supports automated 2-pass encoding.
- It would easily reimplace MPEG2AVI with a extented command line support.

With ifo parsing and subtitles support we have a FlaskMPEG clone, except it will be faster, more reliable (even if Xmpeg 4.2a is good), supporting AC3, and able to frameserve much better.

As for SVCD/XVCD users, SAVE frameserve capabilities should make it a must have .

Developpers should give their opinions and what they want SAVE to be.

[philippas]

Excellent work guys !!! Waiting for a public release :)

@ Slyde
Ifo parsing is sure a neat option but not a must have, because you can easily achieve it by using smartripper with the option stream processing.

[Slyde]

philippas => I know that but displaying in DVD2AVI the stream langages would be nice.

[int 21h]

Originally posted by Slyde
It is already more than a frameserver as you won't need VirtualDub anymore with all features I listed implemented. I think it should become for AVI users the fastest standalone AVI encoder as :

- It does resize/cropping.
- It will for sure do audio encoding/AC3 mux.
- It does frame range selection.
- It supports automated 2-pass encoding.
- It would easily reimplace MPEG2AVI with a extented command line support.

With ifo parsing and subtitles support we have a FlaskMPEG clone, except it will be faster, more reliable (even if Xmpeg 4.2a is good), supporting AC3, and able to frameserve much better.

As for SVCD/XVCD users, SAVE frameserve capabilities should make it a must have .

Developpers should give their opinions and what they want SAVE to be.

Whatever we do, we can't change too much in the way of things like dvd2avi project file format. Because obviously we would lose compatibility with Mpeg2dec.dll. As far as frameserving capabilities, I don't know about others, but I myself use Mpeg2dec.dll via Avisynth much more than I use DVD2AVI to do any actual frameserving.

[Julien]

A little suggestion... is it possible to define the translation in a separate file ? like this :

In English :
_OPEN = Open
_SAVE = Save
_SAVEAS = Save as...

etc.

In French :
_OPEN = Ouvrir
_SAVE = Enregistrer
_SAVEAS = Enregistrer sous...

It would be great ! SAVE could be translate on the fly and it will be easier to translate ! Everybody can translate SAVE without having knowledge in programmation :)

[int 21h]

I could get more of the project setup within Sourceforge (which I've utilized before) if I had Project Manager status. I don't necessarily want to be Project Manager, but I would like to use the Sourceforge facilities to their fullest extent (Forum usage, Tracking tasks, etc).

:)

[Julien]

Done. Thanks for your help :)

[int 21h]

No problem.

Ok, general public type people should do all of the 'feature' requesting here: http://sourceforge.net/forum/?group_id=44406

Developers on the project will also see a developer forum listed there, I suggest we do all further discussions there.

General public type people should also check the Sourceforge page semi-regularly if they are excited about release, because that is the first place it will be listed.

[int 21h]

Developers will also notice that I have setup some of the tasks in the Tasklist. This will enable us to track the status of modules and current code developments.

Please be aware, for this system to work effectively, we will all need to know how to check in our latest code changes.

The task list is divided into two subprojects, the first is Features, which contains the tasks that all either augment or add a new feature to the original DVD2AVI codebase. (AC3 muxxing, IVTC, etc.) The colors indicate priority, grey is lowest, the reddish color is the highest. The other subproject is to contain code base changes that refer to fixing DVD2AVI code bugs in original AC3 code, Project code, Forced Film, etc.

Feel free to request a certain project exclusively in the Developer forum.

[trbarry]

Well, there's nobody over there yet so just for the moment I'll add my comments here as they sort of fit in this thread.

My own priorities are a little different because I also use DVD2AVI for HDTV captures. But for now, while I won't necessarily code all of them, I'd really like to see:

* Somebody's 2-pass support

* AC3 passthrough with good sync and no decoding for reasons I stated above. Generally more support for keeping sound in sync somehow.

* More Robustness. DVD2AVI is currently very vulnerable to garbage data. On an analog capture that makes ugly pictures but on a digital capture or DVD rip it crashes it. My WinTvHD card is very good at making garbage data. When we have an obviously bad frame or trapped error we should probably repeat the previous frame as needed to keep things in sync, then continue. I don't know about obviously bad sound.

* My SimpleResize filter (www.trbarry.com/Readme_SimpleResize.txt). I don't know how similar it is to Bresneham (I thought that was a line drawing algorithm but I get the idea) but it is faster and less soft than Bilinear and I already wrote it. And I'll be adding more SSE2 and SSEMMX optimizations.

* IVTC/Deinterlace (anybody's, not necessarily my GreedyHMA)

* A way to get the improvements into mpeg2dec in an enhanced and compatible fashion. We may want to invent a new .d2v file extension and the ability too add a few fields to those files if we otherwise become incompatible. This in turn would mean maybe a new name for a version of mpeg2dec that could read them. But there is so much shared code it would be a shame to not let the frameserved stuff take advantage of the improvements. I also sometimes like to use Avisynth.

* More performance improvements. I wanted to stop at a stable place and get this out to CVS but I still don't have all of Dmitry's code/suggestions in.

* Someday in the future maybe the ability to read ATSC transport streams. Converting is a pain and each additional copy takes 8.5 GB / hour on my drive. But I'll maybe wait until there is more than just me wanting this one.

Thanks again to Julien & int 21h for getting CVS set up.

- Tom

[Julien]

Here is a little (very little) an crappy explanation on how to use TortoiseCVS (in French) : http://www.media-video.com/forum/viewtopic.php?p=7131#7131

Please correct me when I am wrong (I make a lot of mistakes ! :D )... It can help but be carefull because I really don't understand how does it work !!

Good luck and thanks to int 21h

[int 21h]

The best documents on using Sourceforge and CVS in particular, that I know of, reside here: https://sourceforge.net/docman/display_doc.php?docid=766&group_id=1

If you follow those directions exactly (except in Win2k, making a seperate batch file for the environment variables, run that batch before doing the exe stuff), it will work perfectly.

Then to checkout a copy of the code, or submit one of your updates:

5. To checkout a copy of the latest source use this command:

cvs -d:ext:username@cvs.projectname.sourceforge.net:/cvsroot/projectname
co projectname
After the initial checkout, you can change into this directory and execute cvs commands without the -d option. For example:
cvs update
cvs commit -m "comments for this commit"
cvs add myfile.c


In any event, once things start moving along, everyone will want to issue the update command often, that will run through and find modified files and update your local copy to be current with the CVS copy. If there are any conflicts, like if I and Tom changed the same line of code, and I commited it before he did, CVS would let him know, and mark the conflict code in his file (when he issued the update command). Then Tom could go through the code, figure out what is current, change it, save it, and commit the new modified file. Then when I updated the next day, everything would be fine. There's tons more info if you've never done this before, its available here: http://cvsbook.red-bean.com/cvsbook.html#Detecting_And_Resolving_Conflicts . If you drop off the anchor on that URL, you can get to the contents of that great book.

Oh and last thing, make sure you're using down and dirty commandline versions of both SSH and CVS. GUIs are nice and all, but believe me when I say, its WAY easier to setup the commandline and update/modify from the commandline then it is from those sometime goofy GUIs.

Good luck.

[trbarry]

Oh and last thing, make sure you're using down and dirty commandline versions of both SSH and CVS. GUIs are nice and all, but believe me when I say, its WAY easier to setup the commandline and update/modify from the commandline then it is from those sometime goofy GUIs.

Spoken like a true programmer. ;)

But I've had good luck with WinCVS and SSH on DScaler and now on save-oe, now that it's set up. I maybe don't have the power and control of the command line interface but I can check things in and out, etc.

Any of them work but it looks like we know how to do it about 3 ways now that there is really CVS stuff to checkout. So I'll probably stick with WinCVS just so I don't have to learn something new.

- Tom

[int 21h]

Originally posted by trbarry

But if you have those Intel manuals now, can you figure what the cache line size is. I've seen it both as 64 and 128 in different Intel manuals for the P4.


Intel Pentium 4 and Intel Xeon Processor Optimizations p. 6-3 "Optimizing Cache Usage for Intel Pentium 4 Processors"

The prefetch instruction, inserted by the programmers or compilers, accesses a minimum of one cache line of data (128 bytes on the Pentium 4 processor) prior to that data actually being needed.


:)

[Bruce]

Well, well ! It's been a while I haven't read the doom9's board (too much work to do and so less time... :/).

Slyde told me about SAVE so I wanted to have a look here. Guys, what you are doing is amazing, I just can say "CONTINUE" ! :D

About the release, Slyde told about a command line, this would be good for "rip-pack" such as mine or others (DVD:reaper for example). But I was also thinking about a .dll version, this would benefit from all the windows threaded architecture and would be very easy to implement in any GUI :) Sorry if you already planed that, I've read the 7 pages quite fast ;)

(and a little word for all the froggy's here, les français ils rulez le DivX :D).

[int 21h]

Essentially, Mpeg2dec.dll is already a dll of DVD2AVI.

[trbarry]

@int 21h -

Do you have any idea of how current the mpeg2dec code is in the save-oe project? I got the impression that developement had diverged somewhat.

- Tom

[Bruce]

Originally posted by int 21h
Essentially, Mpeg2dec.dll is already a dll of DVD2AVI.

You're right... How could I have forgot that ;)
But mpeg2dec.dll doesn't includes all the HUGE features SAVE will (re-size, crop, deinterlace, Inverse Telecine...)

[int 21h]

@Tom:
As far as I know, the code in mpeg2dec.dll is fairly current and inline with the last official release of DVD2AVI which is 1.76. The only differences that I've seen from glancing over the two ( I didn't do any complicated comparison procedures or anything ) were class changes and some variable changes that allow for it to be encapsulated into the .dll. The only code added into Mpeg2dec.dll that doesn't exist in DVD2AVI are the filters that dividee added.


@Bruce:
I think it would be fairly trivial to integrate alot of the code changes from DVD2AVI into mpeg2dec.dll

int 21h

Originally posted by trbarry
@int 21h -

Do you have any idea of how current the mpeg2dec code is in the save-oe project? I got the impression that developement had diverged somewhat.

- Tom

[A_Pleite]

You may want to include this tool in save, It could give save the status of somekind of standalone-divx4-nandub.
http://divx4log.narod.ru/

A_Pleite

[BlackSun]

Hi Int21h, want to apply PowerDivX 4 beta test ? now that I know who you are :)

[int 21h]

Originally posted by A_Pleite
You may want to include this tool in save, It could give save the status of somekind of standalone-divx4-nandub.
http://divx4log.narod.ru/

A_Pleite

DivX 4 log editing has proven to be unreliable and ineffective at best due to numerous DivX 4 code methods. You can ask around, but I'm fairly certain most serious accurate log editing methods were abandoned because of lack of support by the DXN team.

[trbarry]

DivX 4 log editing has proven to be unreliable and ineffective at best due to numerous DivX 4 code methods. You can ask around, but I'm fairly certain most serious accurate log editing methods were abandoned because of lack of support by the DXN team.

But I wonder if that would also apply to XviD.

- Tom

[int 21h]

Originally posted by trbarry


But I wonder if that would also apply to XviD.

- Tom

It doesn't, but since XviD is using the Nandub stats format, you could use Gknot for stat file editing, and borrow Nandub code to implement stat file usage in XviD itself.

Regretably, I am not experiencing as much free time to work on this project as I would like, hopefully I get some more free time before the long stretch of free time this summer. :)

[trbarry]

I haven't really done anything on this recently either, but I always run hot & cold on stuff like this. Eventually I'll get a big burst of energy and hopefully do something use full.

If you haven't put them in, could I have the other contributed code optimizations that someone sent in?

- Tom

[Pko]

For the framserve part (the one I use most), I've seen that most people uses it in .avs files... I think that mpeg2dec.dll is mainly a way to include a .d2v in a .avs

I've seen also that many of the improvements already exist in some form as .avs plugins

I've seen also that some people thinks that will be necessary to use a new format since .d2v will not suffice...

Well, why do not include avisynth into save?

It will replace avisynth as .avs handler, with all their functionality (but accepting MPEG files directly), and will include also a new format (.sve?) that will be an hybrid .avs/.d2v in which you can include plugins for avisynth but in a visual way (something a little like virtual dub, but with much more optimized frameserve capabilityes, and you do not need to start vdub, load settings, start frameserve but just to directly open the .sve file as you open a .d2v today)

Just an idea...

[Logos]

I've quickly read this thread and it seems that some of you have problems to access the CVS repository of SAVE, and I've also seen some questions about CVS itself. I'm not a CVS expert, but let me share my own experience and poor knowledge:

In CVS terms the repository is a folder in the filesystem where some projects/modules are archived. Deciding to use one repository for each project or one repository for many projects is up to the system admin. In the case of SourceForge, there is one repository for each project (from the user's point of view ;)).

Now, WinCVS: just install it with all options (if any, I don't really remember). Note that you DO NOT have to install Tcl: WinCVS will tell you that Tcl isn't installed but you really don't need it.

In WinCVS choose first the location where the "save-oe" folder will be created upon extraction: Use the menu Window/Browse Location/Change... (I've choosed D:\projects for example)

To extract files from CVS, we first have to login to the particular CVS repository: press Ctrl+F1 to bring the preferences Window. Now in the general tab, enter anonymous@cvs.save-oe.sourceforge.net:/cvsroot/save-oe in the CVSROOT field, choose "passwd" file on the cvs server for the authentication method, select cvs 1.10 (standard) for the cvs version to use. In the Globals tab, check use TCP/IP compression and set the compression level to 9. Finally click on the OK button. You've configured your cvs access.
Now you have to login (menu admin/login...): just press enter when you're prompted for a password. Wait a few secs, and if everything goes well, you're logged into the server.
The last step: checking out the module save-oe:
In the Create menu, select Checkout module.... In the dialog box that pops up, enter save-oe for the module name then click OK and the download begins.... That's all

There is a more straight forward way for just anonymous check out, but this one applies also to developers: you've just to replace "anonymous" by your own login in the connection string, and enter the right pwd when prompted.

And YES: we've got a module (folder) called save-oe in a repository called save-oe too. That's not an error :)

Now that I've fetched save-oe's sources (merci les mecs ;)), I'll try to see what I could implement myself. I've thought about some sort of cluster encoding (with multiple machines) ;)

Logos.

[stoney]

did this project die?

[bb]

I'm interested in the project's status, too.

bb

[trbarry]

I'm not sure it ever quite got started. It was probably a very bad idea suggesting above that developer messages be moved to the mini-forum on SourceForge since there never was sufficient interest there to keep it going.

I've been devoting more of my efforts recently to Xvid and some other things and I don't know if anyone is actively working on the save-oe project. I know there is zero activity on Sourceforge.

I think a project like this needs a champion to keep folks interested and organized.

- Tom

[macdaddy]

I've been devoting more of my efforts recently to Xvid
That's great news.

[trbarry]

----------------------------------------------------------------------
I've been devoting more of my efforts recently to Xvid
----------------------------------------------------------------------

That's great news.


You mean it's great news for DVD2AVI? ;)

- Tom

[Guest]

@trbarry

LMAO

[macdaddy]

great news for XviD...

you haven't been playing with it as a capture/television-archiving codec have you..?

I thought I remember that time-shifting was something that was being tossed around as an add-on for dscaler...

[trbarry]

macdaddy -

My original interest in XviD was for HDTV caps. But that got me interested in tuning it because it's hard to play HDTV in software.

There are a few people playing with putting capture & time shifting in DScaler but I'm not one of them and I think it is still very experimental.

For me the cross-over between XviD and DScaler, if any, is more likely to be:

1) Maybe putting Greedy/HM deinterlace into XviD

2) Maybe making a DScaler XviD playback filter. I'd only do this if I thought it could be a lot faster than Dshow. (HDTV again)

And I see I'm way off topic here. Sorry.

- Tom