Command Line Compressor - AVS2AVI

[DSPguru]

@int 33
my mailbox is full, please post your request in here.

[int 21h]

Your terms sound fine.

Would it be possible to change the output of the .dll so that when its deinitalized it does not show the URL message? I would be more than willing to show the URL in the beginning, at the end it really messes up our program's output (because it has more to do after its done with the audio).

Right now our program startup looks like this:


AVS2AVI v1.15 (c) 2002 Christophe PARIS (christophe.paris@free.fr)
Contributions by: int21h

Thanks to Avery Lee author of VirtualDub and Chia-chen Kuo author
of DVD2AVI for providing great source code of inspiration.

Also uses:
BeSweet v1.4 by DSPguru.
--------------------------


I think it would be good as:


AVS2AVI v1.15 (c) 2002 Christophe PARIS (christophe.paris@free.fr)
Contributions by: int21h

Thanks to Avery Lee author of VirtualDub and Chia-chen Kuo author
of DVD2AVI for providing great source code of inspiration.

Also uses:
BeSweet v1.4 by DSPguru. Check for the latest version at: http://dspguru.doom9.org


Because when it outputs the URL at the end it looks very, very bad. And also, do you intend to fix the info-> bug?

Thanks again.

[mandm]

[QUOTE]Originally posted by Emp3r0r
[B]Well, everyday things are getting easier. Most of the hard stuff is already done. The way I see it, it is now as simple as:

1) Select your DVD Drive
2) Select the streams
3) Get besweet settings or DSC source sound
4) Create avisynth script
5) Get codec settings
5) Run avs2avi

...

What's neat is that all these things get better everyday making everyone's life easier, including whoever creates the next great gknot.

[Back on topic]

how do you do 1-2 ???

about the program we were thinking to evolve M&M to include advanced functionalities as besweet...

[DSPguru]

#ifdef BeSweetDLL
if (!strcmp(*argv,"singleline"))
fprintf(stderr,"BeSweet %s by DSPguru. Check for the latest version at: http://DSPguru.doom9.org\n",BeSweetVer());
else
#endif
pinfo(1);


i will check out the info bug asap.
btw, i believe this bug only relevant for wav source. i believe it works fine for ac3 source.

ps,
it would be very cool if we could have muxing in your tool. i would really like to see a tool that mux BeSweet's payload directly into a video stream. this will save precious i/o time.

[int 21h]

Where should I place that code (I dont ever #define BeSweetDLL so I don't think this would do anything)? Also, what does pinfo(1); do?

[DSPguru]

this is a BeSweet code.
pinfo(1) is what BeSweet have today - the two lines greetings.
as you can see from code above, if argv[0] will be "singleline", BeSweet will print a single greetings line instead of pinfo(1).

[int 21h]

Ah, thanks. This is updated in the dll, so I should download a new copy, or this already existed? :-P

[DSPguru]

no new binary yet :o

[int 21h]

This change also makes it so it omits the URL output when its finished processing right?

[DSPguru]

#ifndef BeSweetDLL
fprintf(stderr,"\n\nVisit DSPguru's Homepage at :\n");
fprintf(stderr," http://DSPguru.doom9.net/\n");
#endif

[int 21h]

Really? :( You won't consider changing that if we already mention the URL in the beginning?

[DSPguru]

reread :
ifndef :D.

[int 21h]

Gah, sorry, was reading quickly before I go out for the evening.

Thanks again.

[[Toff]]

it would be very cool if we could have muxing in your tool. i would really like to see a tool that mux BeSweet's payload directly into a video stream. this will save precious i/o time.

Yes, sure it would be cool.
But what about the AVI problem with VBR files ?

I don't know much about muxing myself.

[MaTTeR]

Originally posted by [Toff]
Yes, sure it would be cool.
But what about the AVI problem with VBR files ?
Why not use the Ogg container(OGM)?

[[Toff]]

MaTTer : Why not use the Ogg container(OGM)?

In a program called avs2avi ;) it's not possible.
You will need an avs2ogm :D

Is it possible to mux AC3 in OGM now ?

[MaTTeR]

Originally posted by [Toff]
[B]Is it possible to mux AC3 in OGM now ? LOL, well you do have a point:D

Yes, I've been muxing AC3 into OGM for quite sometime now with VdubOGM which is now incorporated into VirtualDubMod I think.

http://forum.doom9.org/showthread.php?s=&threadid=33804

[int 21h]

Well, I've never had any problems with the fact that muxing a VBR audio stream into an AVI produces 'invalid' AVI files. It has never failed to play correctly for me... so I continue to use it.

That being said, if anyone wishes to submit mods to allow the usage of OGM in AVS2AVI, I'm sure they would be accepted ;) However, I won't be coding for it until I figure out all the AVI stuff.

[TheUnforgiven]

thanx guys for the great program.
i already use it as well in my GKnot like program QuickAVS:
http://forum.doom9.org/showthread.php?threadid=37626
now small request:
can u plz add a pause after the program finishes processing so that one can read the average fps value before the dos window closes.

[int 21h]

What language is your program written in? In all honesty I think a better solution would be for you to redirect stdout to a window or something in your program. If you give me more details about your program I can help you accomplish this, and it will be much better looking than a dos box opening up :)

Also, once things get stable, and I either figure out, or give up on muxxing, I can probably be persuaded to make a .dll for avs2avi.

[TheUnforgiven]

thanx for the responce int 21h
the program is written in vb6. i think u mean something like the right window in headac3he? this sure would be better. is this done with win32 api functions? if yes i will be grateful if u gave me the names of these functions and i'll search the MSDN for more info.
the dll would be great also, but i would like to know how this is done even if there will be a dll.
good luck on muxing

[int 21h]

http://support.microsoft.com/default.aspx?scid=KB;en-us;q150767

http://www.codeproject.com/atl/redirect.asp <-- good.

[TheUnforgiven]

@int 21h
thank u for the links i'll try to implement that in the next version

[iago]

@[Toff] / int 21h

Thanks a lot for this tiny but great command-line program! ;)

regards,
iago

[Koepi]

Now only a small GUI is missing, for users convenience ;) (just showing up if no command line parameters are given).

Maybe I could do a small MFC GUI directly using your code, I think it would be a nice vdub replacement... ;)

Regards
Koepi

[TheUnforgiven]

@koepi
my program QuickAVS offers a GUI already have u tried it
http://forum.doom9.org/showthread.php?s=&threadid=37626

EDIT: sorry i just reread ur post u mean from inside the program.

[int 21h]

Any news on a new .dll release DSPguru?

[DSPguru]

i'm very very very very very very busy lately. and again, this is only a plastic bug that relates to wave transcoding. so you'd have to wait.

ps,
feedback on ac3 transcoding is more than welcomed.

[int 21h]

all I really need is the ifdef, and ifndefs, the bug can wait....

[DSPguru]

it's up.

[int 21h]

I'll take a look, and post the new version of this today :)

[int 21h]

With the new callback function in the .dll it should make muxing a little bit easier, I just have to figure out how to convert the data into a stream for the file...

[DSPguru]

great :)

[int 21h]

To break the 4GB barrier?

[mfluder]

Hi,

I made a little GUI for this great program. It's very small (only 16K) and a good thing about it is it wont immediately close console window after encoding is finished. Instead it will wait for you to press a key, so you will be able to see average fps and encoding time. If you have some strange things going on while using avs2avi 1.14 (like strange output during encoding) then please download avs2avi 1.13 until this "bug" is fixed. As I understand int 21h already fixed this but he will not release it until there are some bigger changes, like audio support. I almost forgot to mention that you will have to put this file in same directory with avs2avi.exe.

I attached a file to this post but you will have to wait until it is approved. Please report bugs if you find any.

@[Toff] and int 21h:
Thank you very much for this great tool.

mfluder

[octapus]

This little program is very helpful. And using it, i thought of a feature that would be very useful, but i don't have a clue how easy it is to implement or if it is even possible to be implemented.

How about having the possiblity to pause encoding, close program ,reboot and after computer restarts, to continue from where (frame number) it had left encoding (having an option to do so).

Any ideas, everybody? I just need to hear some opinions on how this can be implemented. The only thing that i can think of and which maybe is far from what i want, is "hibernate" option of windows xp(the way it works).

Thanks.

[DaveEL]

Originally posted by octapus

How about having the possiblity to pause encoding, close program ,reboot and after computer restarts, to continue from where (frame number) it had left encoding (having an option to do so).


Yes this is possible. I know ive done it before (lost the source code now and the thing wasnt very useful as it was more a proof of concept rather then a real implementation some things needed to be done by hand). My previous implementation relied heavily on features in nandub and gordian knot but it should be possible to do without these however file size predictability will not as good (my old implementation only dropped changes the file size by a couple of kbytes (undersized) each time you paused (and only in the second pass first pass was losless). Also my old implementation could cope with machine crashes (i even puilled the power cable out of my pc a couple of times to test this) but without some of the nandub features i used i dont think this will be possible.

Do you want to use this for one or two pass encoding? one pass is very easy but two pass should be possible (and far more useful) but will be limited to certain codecs (i would write it for xvid but the code would be available if someone wanted to write divx support). Also i need to work out some issues concerning b-frames as vfw codecs were not designed to do b-frames and it could cause problems with my intended implementation.

Ill try to work out if this is really practical and if i have time to do it soonish. Also im not sure if i can do this with avs2avi anyway it may need to be rewritten to allow some extra functionality that this would require.

If anyone else wants to have a go at writing this pm me and ill explain how i implementated this last time and my concerns about doing this without nandub/with b-frames etc.

DaveEL

[octapus]

DaveEL:


That's great news. I have been waiting for such a feature for a long time. Since you implement according to codec used, i use only xvid dev-api3 with virtualdub or avs2avi. Of course two pass is desired, but if it is trouble for you, one pass is a step forward, also.

It would be even better than my thoughts if it could handle power supply loss. I say this because what i asked was simply the user to pause,resume encoding (resume even after computer restart) supposing that we have constant power supply.

If you want me to help with coding i could give a try, but i'm a new c++ coder(learned it previous semester) and haven't codec anything serious so far.

Again, thanks a lot.

[int 21h]

Just to let everyone know how development is progressing, I've been real busy lately outside of this realm, and so development has slowed somewhat, this next week looks relatively clear though, so I am hoping on getting some work on the code done then.

[[Toff]]

DaveEL, octapus:
Any progress with encoding pause ?
As DavelEL says, one pass encoding should be easy and can work for nearly every codec. If you look into avs2avi, after encoding a frame you have a KF flag. When you want to pause encoding you can wait for the next KF (hey you don't want to loose any processing ;) then close all the file cleanely (as with a Ctrl+C).
Then on resume, instead of creating a new avi, you open the precedent file, you check its length and go to its end, you seek to the same frame in avs, and encoding can continue.

2 pass encoding, is codec dependant so you need to save specific information.

[DaveEL]

Originally posted by [Toff]
DaveEL, octapus:
Any progress with encoding pause ?
As DavelEL says, one pass encoding should be easy and can work for nearly every codec. If you look into avs2avi, after encoding a frame you have a KF flag. When you want to pause encoding you can wait for the next KF (hey you don't want to loose any processing ;) then close all the file cleanely (as with a Ctrl+C).

Ok 1-pass support kind of works. If you press ctrl-c it now waits for the next key frame before it exits. Note down the number of frames encoded in the output. To resume use the same input file and a new output filename and use -f <number of frames +1 >. The files then need to be appended with virtualdub. Anyone know of a small utility to do this?


http://www.ecs.soton.ac.uk/~del100/avs2avi.zip

DaveEL

[[Toff]]

Great! I'm going to test it right now.
It should be possible to read the number of frame of the 1st file.
And maybe continue with this file.
I will test if it's possible.

[[Toff]]

Here is an new version.

You use the resume function this way.
>avs2avi mymovie.avs xvidmovie.avi -r
Now you can press Ctrl+C at any time during encoding.
It will create a "resume.codec" file that contain codec parameters.
Then you do what you want, stop your computer and go sleeping :D
When you want to continue encoding you do :
>avs2avi mymovie.avs xvidmovie.avi

Voilà !

One more precision, it works only for 1 pass encoding.

AVS2AVI v1.15 (http://christophe.paris.free.fr/temp/avs2avi_115.zip)

Have fun !

EDIT : Whou ! very synchro ;)

[DaveEL]

It should be possible to read the number of frame of the 1st file.


Was going to implement it so i saved this data along with codec config so you could continue where you left off.


And maybe continue with this file.

Tried this a lot and gave up for now. Made it write to the same file but depending on how it was done it would only let me play the first or last parts of the encoding. From what i read of the API docs this is not possible to do with avis. What i did think might be possible but again never got to work was:

1) encode first past to file x.
2) when you restart the encode move x to y and open it
3) create a new avi at x
4) copy each frame from y to x
5) now start reading from the input file again and put the encoded frame on the end of the file y.

Problems
1) couldnt get it to work at all
2) can see any way in the avi to find out if a frame i read was a keyframe so i cant specify this flag in the output file either. The nearest example in the microsoft code instead set the keyframe flag on every frame.


Think i might give up trying on the append stuff for now and work on a) making b-frames work
b) two pass support (xvid)

DaveEL

[DaveEL]

Originally posted by [Toff]
Here is an new version.

You use the resume function this way.
>avs2avi mymovie.avs xvidmovie.avi -r
Now you can press Ctrl+C at any time during encoding.
It will create a "resume.codec" file that contain codec parameters.
Then you do what you want, stop your computer and go sleeping :D
When you want to continue encoding you do :
>avs2avi mymovie.avs xvidmovie.avi

Voilà !

One more precision, it works only for 1 pass encoding.

AVS2AVI v1.15 (http://christophe.paris.free.fr/temp/avs2avi_115.zip)

Have fun !


Very nice i guess i should start working on b-frames and 2-pass support now.


DaveEL

[/i]

EDIT : Whou ! very synchro ;)

Just dont try b-frames things WILL go wrong. Does anyone know the address in the divx5 config i need to look at to check if b-frames are enabled?

[[Toff]]

Just dont try b-frames things WILL go wrong. Does anyone know the address in the divx5 config i need to look at to check if b-frames are enabled?


What's the problem with b-frames ?
If you always cut on a K frame there shouldn't be any problem ?
It works well here with xvid-20112002 and Divx 5.


Here is a possible improvement :
We are waiting for next KF to "pause" encoding. I think that it would be possible to stop encoding instantely.
Then to resume, you reopen the destination file and go to the last keyframe (with AVIStreamPrevKeyFrame) of the file and continue at this point.
I don't know if old unused data would be well overwritten. But it's something to try ;)


Also how you plan to support 2 pass ?
(thinking for myself here)
- For the 1st pass you need to append to the stat file. Maybe done by creating 1 stat file per "session" and appending stat file at end.
- For 2nd pass I think it's not possible to save all data used for file size estimation. When you want to resume, you can erase the beginning of the stat file, and you will have to substract the size of the current destination file to the estimated end file size.

I wonder if adding support for this directly in xvid wouldn't be easier ?


EDIT:
Just another idea I've got :
What would be great is to had a function pointer into the compressor struct of the Xvid codec.
This function pointer will be at a fixed place in all Xvid version so you have no update problem.
The function could look like this :
int XvidCommand(char* command);

you can use it like this (bitrate example) :
XvidCommand("bitrate=900");
int bitrate = XvidCommand("get_bitrate");

or like this (stat file example) :
XvidCommand("stat_file=C:\\mylog.stat");
char* stat_file = XvidCommand("get_stat_file");

or even :
XvidCommand("save_state_file=resume.xvid");
XvidCommand("save_state");

and to resume session :

XvidCommand("save_state_file=resume.xvid");
XvidCommand("load_state");

hmm, just an idea.
In fact it looks a bit like UCI ;)

[octapus]

I just saw that you have incorporated the features mentioned into new version just released. That's good news. I'm going to test it and report back.

Thank you.

[DaveEL]

Originally posted by [Toff]
What's the problem with b-frames ?
If you always cut on a K frame there shouldn't be any problem ?
It works well here with xvid-20112002 and Divx 5.
[/i]

Try a very high max b-frames setting in xvid. Basically avi and b-frames = not good. Each encoding starts with some number of blank frames where the codec was getting enough data for the b-frame encoder. When doing this in several parts this will happen several times.
Also the output is not in sync with the input so you have to restart the source some number of frames earlier.

Divx5 packed bitstreams seems a better solution to the problem then the current xvid support i hope someone will fix the packed bitstream in xvid soon im not sure its really possible to support resuming without it.


Here is a possible improvement :
We are waiting for next KF to "pause" encoding. I think that it would be possible to stop encoding instantely.
Then to resume, you reopen the destination file and go to the last keyframe (with AVIStreamPrevKeyFrame) of the file and continue at this point.
I don't know if old unused data would be well overwritten. But it's something to try ;)

As long as it will write to the correct place in the output this will work fine. The next keyframe was just the easy way to do it.


Also how you plan to support 2 pass ?
- For the 1st pass you need to append to the stat file. Maybe done by creating 1 stat file per "session" and appending stat file at end.

This is also made a bit more complicated by the "last" keyframe thing above. You need to seek back the correct number of frames in stats files as well.

- For 2nd pass I think it's not possible to save all data used for file size estimation. When you want to resume, you can erase the beginning of the stat file, and you will have to substract the size of the current destination file to the estimated end file size.

Yeah thats how it works the trouble is getting the file size predictability back up to the same level it was without resuming as any the internal codec information is lost.

[i]
I wonder if adding support for this directly in xvid wouldn't be easier ?

Just another idea I've got :
What would be great is to had a function pointer into the compressor struct of the Xvid codec.
This function pointer will be at a fixed place in all Xvid version so you have no update problem.
[/B]

To help support Vidomi the first 4 or 5 fields were put at fixed memory locations some time ago. This includes all the important stuff like the mode (1st/2nd pass), output file size and stats file name. So its about the same amount of work to do it in avs2avi as in xvid.

DaveEL

[[Toff]]

Try a very high max b-frames setting in xvid. Basically avi and b-frames = not good.

Yes, I just tried with a Max B-Frame = 8, and there is synchro problem.


So its about the same amount of work to do it in avs2avi as in xvid.

Yes, but if you do it in xvid you can save some internal value. And have easier access to stat file, seek in them.
I will try to hack something in xvid.

[DaveEL]

Originally posted by [Toff]
Yes, I just tried with a Max B-Frame = 8, and there is synchro problem.


Yes, but if you do it in xvid you can save some internal value. And have easier access to stat file, seek in them.
I will try to hack something in xvid.

That can be done but you can get very accurate encodes if you just manipulate the output size correctly (the version i wrote with nandub was never more then a few k out).


Been trying to hack on b-frames (divx 5, would prefer xvid but divx b-frames are better for now). Have the resuming working now i just need to fix the existing bug with avs2avi with b-frames.

Try this take a short clip and encode with b-frames in both virtualdub and avs2avi compare the two frame by frame and you will notice the one from avs2avi is one frame out.

DaveEL