Command Line Compressor - AVS2AVI

[iago]

Thanks everybody putting effort on this great tool! It's really fantastic! :)

regards,
iago

[DaveEL]

Originally posted by MaTTeR
On my AMD dualie the speed seems and CPU usage seems to be almost on par with VirtualDub. I haven't had a chance to do any "real" benchmarking yet but first results seem very promising to say the least. Excellent work guys!

Can you try an old version (anything before i added the SMP optimisations in that post you quoted) against the latest builds.

The only problem I'm having with a few of the versions here is that avs2avi is locking up during the first pass for some reason. CPU usage reamins very high but no encoding is actually taking place according to Debug View.

Does it lock straight away or does it encode a bit and then lock later?

At some point during the 1st pass of XviD the CLI is reporting bogus ETA and FPS information, at the start of the encode it seems to be just fine though. I'll report back more after some further testing this week. Many thx again for all the efforrts here:)


FPS/ETA is based on the average speed not the current speed. That could be why it looks wrong.

DaveEL

[bilu]

I'm getting output related to deviation in avs2avi with XVID builds after 09 December, both Koepi and Nic's. Is this related to B-frame compensation? I'm @work so I can't send you any output now.

Another question:
I make OGM movies using AVS2AVI, BeSweet and OggMux.
Since I use B-frames that should be compensated for the AVI stream by AVS2AVI,would this afect the OGM movie? I've encoded a movie (not seen it, was ripped in my HD) and noticed speaking delay in some parts in the middle, but not in the whole movie. Still I don't know if the delay was already present in the source DVD.

[DaveEL]

Originally posted by bilu
I'm getting output related to deviation in avs2avi with XVID builds after 09 December, both Koepi and Nic's. Is this related to B-frame compensation? I'm @work so I can't send you any output now.

Another question:
I make OGM movies using AVS2AVI, BeSweet and OggMux.
Since I use B-frames that should be compensated for the AVI stream by AVS2AVI,would this afect the OGM movie? I've encoded a movie (not seen it, was ripped in my HD) and noticed speaking delay in some parts in the middle, but not in the whole movie. Still I don't know if the delay was already present in the source DVD.

Checked the changelog for xvid around the 9th and can't see anything but i do remember a cvs change the other day to adjust the 2 pass code for the new version of virtualdub. Can you check the latest builds at http://xvid.hopto.org/ and tell me if these present a problem. Also are you using the resuming support with 2-pass/b-frames this will be broken until i understand the new stats file format. How big is the size deviation? the b-frame code in avs2avi should only throw the size out by a few bytes*max bframes, but if the 2-pass code is adjusting for differences in way virtualdub does b-frames this would explain it. If this is the case ill need to change the b-frame code to do the same as vdub.


Not sure about OGM issues i assume it just remuxes the frames in the avi so any problems in the avi will be present in the ogm. Try using the packed bitstream option (warning this is still buggy) to remove the delay in the b-frames. Unfortunatly until this option is fixed b-frames wont work properly.

[bilu]

I'm using 1-pass CBR, no resume.
I'll send you some output tonight.

[DaveEL]

Hmm the 1-pass code should be fine have you tested with virtualdub to check its not a problems with xvid rather then avs2avi.

DaveEL

[bilu]

I haven't noticed any problem in the video output but I didn't check the filesize at the time. What I have noticed was something related to deviation in the screen output of avs2avi that didn't appear with any of the prior builds of xvid.

[DaveEL]

Avs2avi outputs the size of the bitstream rather then the avi so the size printed will be a little smaller. Older versions i believe printed the size of the file but the buffering code meant the size jumped up every now and then as frames things were not commited to the avi directly.

DaveEL

[bilu]

This is the screen output I get with every XVID build after 09/12.
I use the exact same parameters: 1 BF, ratio 150%, offset 0.


D:\dvd_progs\QuickAVS\filters\tests>avs2avi teste.avs teste.avi -l codec.txt -p
0

AVS2AVI v1.22 (c) 2002 Christophe PARIS (christophe.paris@free.fr)
Contributions by: int21h
B-Frames and resuming support by David Leatherdale (dave@leatherdale.net)

Thanks to Avery Lee author of VirtualDub and Chia-chen Kuo author
of DVD2AVI for providing great source code of inspiration.

Source:
* Filename: teste.avs
* Bit depth: 16
* FOURCC: YUY2
* Frames: 781
* Resolution: 528x320
Compressor:
* Name: XviD MPEG-4 Codec
* FOURCC: XVID
Destination:
* Filename: teste.avi
Pass 1/1 frame 0/781, 0.00fps, ETA: , size: 0.00MBCBR: frame: 0, quant: 5, devia
tion: 14852

Pass 1/1 frame 2/781, 6.08fps, ETA: 00:02:08.145, size: 0.02MBCBR: frame: 1, qua
nt: 5, deviation: 18128

Pass 1/1 frame 3/781, 5.78fps, ETA: 00:02:14.593, size: 0.03MBCBR: frame: 2, qua
nt: 5, deviation: 23438

Pass 1/1 frame 4/781, 5.80fps, ETA: 00:02:14.032, size: 0.03MBCBR: frame: 3, qua
nt: 6, deviation: 23478

Pass 1/1 frame 5/781, 5.75fps, ETA: 00:02:15.023, size: 0.04MBCBR: frame: 4, qua
nt: 6, deviation: 23209

Pass 1/1 frame 6/781, 5.75fps, ETA: 00:02:14.849, size: 0.04MBCBR: frame: 5, qua
nt: 6, deviation: 23179

and so on...

[DaveEL]

ah i forgot about that all xvid debug info for cbr mode is directed to the console this is an xvid bug ive been meaning to report. Dont worry nothing is wrong.

DaveEL

[Emp3r0r]

can we get this data during a dual pass and possibly count the quants and make a nice chart that displays quant frequency?

[DaveEL]

Originally posted by Emp3r0r
can we get this data during a dual pass and possibly count the quants and make a nice chart that displays quant frequency?

Not really somthing that should be in avs2avi but if you want to do it xvid prints this info to the windows debug function so you should be able to get your application recieve these messages looks like a good place to start reading would be here (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/debug/base/about_basic_debugging.asp)

DaveEL

[bilu]

@ Emp3r0r

can we get this data during a dual pass and possibly count the quants and make a nice chart that displays quant frequency?

I've been using CBR and never used 2-pass before, but isn't that what DebugView does?

[DaveEL]

Originally posted by bilu
@ Emp3r0r

I've been using CBR and never used 2-pass before, but isn't that what DebugView does?

Yes debugview captures the debug output that i pointed out above.

[DaveEL]

New version again hopfully this should fix the 2pass resuming support but ive got a lot of work to do currently so its not well tested. It shouldn't stall after the first pass at least when resuming is enabled and shouldnt crash when pressing ctrl-c on the first pass of two pass encodes (everything worked before the second pass stuff was not closed properly causing an error message was all).


http://www.ecs.soton.ac.uk/~del100/avs2avi.zip

DaveEL

[DaveEL]

Ok so i did some testing when i got back from the pub so here is another new version which i really think works this time (but i was drunk while testing so who knows ;) )


DaveEL

PS i would like to thank redbull and vodka for keeping me up long enough to debug this build.

[((( atom )))]

ok [edit] ...oops - wrong thread!

[wotef]

hi, lovely app

bug report: with 03/01 xvid build, the latest version of avs2avi fails to insert any keyframes if b-frames are enabled - virtualdub still manages to do so - if you then open the avs2avi encoded file in vdub and press CTRL+rightarrow (jump to next keyframe), you get a message that reads something like "warning: nothing to output, b-frame decoder lag"

[DaveEL]

Originally posted by wotef
hi, lovely app

bug report: with 03/01 xvid build, the latest version of avs2avi fails to insert any keyframes if b-frames are enabled - virtualdub still manages to do so - if you then open the avs2avi encoded file in vdub and press CTRL+rightarrow (jump to next keyframe), you get a message that reads something like "warning: nothing to output, b-frame decoder lag"

Hmm i get the same thing except i get it when using virtualdubmod too can you retest to make sure virtualdub is working correctly.

Also i just put up a new version with a bframe bugfix.

DaveEL

[cult]

sorry,but where?

[Pasqui]

@cult
the same address as usual:
http://www.ecs.soton.ac.uk/~del100/avs2avi.zip
it still displays AVS2AVI v1.25 but compilation date is different form previous.

[DaveEL]

Originally posted by Pasqui
@cult
the same address as usual:
http://www.ecs.soton.ac.uk/~del100/avs2avi.zip
it still displays AVS2AVI v1.25 but compilation date is different form previous.

whoops sorry about that forgot to increase the version number (i keep the same file name as i dont like keeping buggy builds around or dead links).

DaveEL

[cult]

thank you.I downloaded but I thought it was the same version.All clear now:)

[jonas]

first of all I have to say that this is a great and very usefull tool.

But is the a chance that special charakters like ä for example can be used for filenames as well? If it is complicated just leave it, as it is easy to work around as soon as you know that you can't use those chrakters.
I just got confused when I was testing you app yesterday evening I couldn't get it to work at first.

Secondly, and I'm not sure whether this has be discussed before or whether this only occured to me, I noticed that there is a large time between the finish of the first pass and the beginning of the second, while avs2avi is still using the cpu. Since it didn't display any new lines (like starting 2nd pass) I was just about to quit the app when it suddenly started the second pass....

thanks again for this tool
jonas

[MaTTeR]

Originally posted by jonas
I noticed that there is a large time between the finish of the first pass and the beginning of the second, while avs2avi is still using the cpu. Since it didn't display any new lines (like starting 2nd pass) I was just about to quit the app when it suddenly started the second pass.... Ack!! I guess I haven't been patient enough. I've been thinking the app was in an infinite loop and killing it after waiting 5-10mins.

[Nic]

That large gap used to happen in my dvd2avi, its because its trying to close an incorrect AVI file (the 1st pass file)

A hack to stop it, is to not do the AVIFileClose(...) on the first pass AVI file. Just skip it & leave the file open.

-Nic

[sillKotscha]

Originally posted by MaTTeR
Ack!! I guess I haven't been patient enough. I've been thinking the app was in an infinite loop and killing it after waiting 5-10mins.

*ouch* patience is one of things i don't have and killed it after ~5min, too :D

and the really madest thing is, that after finishing the first pass (I swear i didn't touched anything) WinDVD started :confused:

Sill

[DaveEL]

Originally posted by Nic
That large gap used to happen in my dvd2avi, its because its trying to close an incorrect AVI file (the 1st pass file)

A hack to stop it, is to not do the AVIFileClose(...) on the first pass AVI file. Just skip it & leave the file open.

-Nic


Not sure how that world work with my resuming support which counts the number of frames in the file to find out where to resume from so ive just written some new threading code to start second pass before the file is closed. codes written but have no time to test it now. (anyone who wants to test drop me a messages and ill send you the code or a binary). Ill try to get this done next week when i have done all my coursework.

DaveEL

[DaveEL]

hmm my departments web server seems to be broken try http://www.soton.ac.uk/~del100/avs2avi.zip

DaveEL

[DaveEL]

Originally posted by Nic
That large gap used to happen in my dvd2avi, its because its trying to close an incorrect AVI file (the 1st pass file)

A hack to stop it, is to not do the AVIFileClose(...) on the first pass AVI file. Just skip it & leave the file open.

-Nic

thinking about it would this not be best fixed by making xvid output a valid streram (well valid to the standard avifile routines) in the first pass (i guess if it output 1 bytes frames instead of 0 it would be fine)

DaveEL

[Nic]

Not necessarily as divx5 has the same problem. My solution is what I use & it works but isn't nice. the best solution is to write your own or use another avi output routine (vidomi's code is quite good)...

Infact I guess there is no need to actually create a first pass file at all, just use ICCompress but don't do the avifileopen, avistreamwrite, etc.

-Nic

[jonny]

Yes, what nic says is correct (at least for divx 5), no need to write the Avi in the 1°pass, i've made some real test about this some weeks ago (using avs2avi source :) )

[DaveEL]

Originally posted by jonny
Yes, what nic says is correct (at least for divx 5), no need to write the Avi in the 1°pass, i've made some real test about this some weeks ago (using avs2avi source :) )

Except that in order to do the resuming i need to know what point i encoded up to so i count the number of frames in the file.

DaveEL

[jonny]

You can always write a fake avi: a file that simple contain the number of frames encoded (this file will be written if the user press ctrl-c)

[Nic]

@Dave: Hmmm, but there must be a better way of counting what frame you were upto other than the 1st pass avi. Registry key or file could do?...only a thought though.

Cheers,
-Nic

[DaveEL]

Originally posted by Nic
@Dave: Hmmm, but there must be a better way of counting what frame you were upto other than the 1st pass avi. Registry key or file could do?...only a thought though.

Cheers,
-Nic
Its just that the info in the avi file is already available seems redundant to keep it in a seperate place. While i could save the frame number one small problem is the frame to resume from is not the last frame encoded its the frame before the last keyframe so i would need to keep a seperate count of that (buffering commits frames to the avi when a keyframe is encounterd). However i have come up with a different idea that i can ignore the first pass data (all the 0 length frames) and instead produce an avi with the right number of frames but with data generated by avs2avi which should then save without the delay associated with the normal first pass file close. Also with this way the first and second passes are handled in much the same way and so reduces possible problems (i hope). hmm now all i have to remember is how to do function pointers in c. Besides registery keys are bad if someone does more then one encode at once (why they would want to i dont know but they might. I suppose it might be slightly faster on SMP systems but thats somthing i have an idea that i will try to implement soon should be possible to split up the encode with very little loss of quality if you use external stats processing).


Once again ive got loads of work to do so it may take a while however avs2avi is in general more intresting then my uni work so it may be done soon :).
DaveEL

[DaveEL]

And for my 200th post here :)

http://www.ecs.soton.ac.uk/~del100/avs2avi-test.zip

Can someone test to see if this removes the pause after the first pass bug.

DaveEL

[Beave]

I'm sorry to bother you again, but I somehow want to use avs2avi for my encodes. here is what mit command window sais:


D:\avs2avi>avs2avi test.avs test.avi

AVS2AVI v1.25 (c) 2002 Christophe PARIS (christophe.paris@free.fr)
Contributions by: int21h
B-Frames and resuming support by David Leatherdale (dave@leatherdale.net)

Thanks to Avery Lee author of VirtualDub and Chia-chen Kuo author
of DVD2AVI for providing great source code of inspiration.

Source:
* Filename: test.avs
* Bit depth: 32
* FOURCC: none (RGB)
* Frames: 51
* Resolution: 320x240
Compressor:
* Name: XviD MPEG-4 Codec
* FOURCC: XVID
Destination:
* Filename: test.avi

AVIFileOpen failed to create destination file test.avi : Das System kann die ang
egebene Datei nicht finden.


Any new Ideas?

[DaveEL]

Originally posted by Beave
I'm sorry to bother you again, but I somehow want to use avs2avi for my encodes. here is what mit command window sais:


D:\avs2avi>avs2avi test.avs test.avi

AVS2AVI v1.25 (c) 2002 Christophe PARIS (christophe.paris@free.fr)
Contributions by: int21h
B-Frames and resuming support by David Leatherdale (dave@leatherdale.net)

Thanks to Avery Lee author of VirtualDub and Chia-chen Kuo author
of DVD2AVI for providing great source code of inspiration.

Source:
* Filename: test.avs
* Bit depth: 32
* FOURCC: none (RGB)
* Frames: 51
* Resolution: 320x240
Compressor:
* Name: XviD MPEG-4 Codec
* FOURCC: XVID
Destination:
* Filename: test.avi

AVIFileOpen failed to create destination file test.avi : Das System kann die ang
egebene Datei nicht finden.


Any new Ideas?

1) Try the load defaults button in xvid?
2) try the new avs2avi-test.zip i just uploaded should return an error number in the message can you post that here please.

DaveEL

[Beave]

1) Try the load defaults button in xvid?
I did, no effect. Same error message.

2) try the new avs2avi-test.zip i just uploaded should return an error number in the message can you post that here please.

I have actually used your latest avs2avi-test.zip.

[((( atom )))]

ok, so i am about to try it again.. one thing is not clear to me: when i encode and decide to restart the pc for any reason - i have [enable resuming] checked - how do i actually do the resume? i don't see any button for that?!?

[wertherman]

Illegal Exception at end of first pass with latest build...

[Emp3r0r]

@atom: push ctrl-c

[DaveEL]

Originally posted by Beave
I have actually used your latest avs2avi-test.zip.

I posted a new version when i posted that message download it and run it it will give you an error number in the message without it can tell nothing about whats going wrong.

DaveEL

[DaveEL]

Originally posted by ((( atom )))
ok, so i am about to try it again.. one thing is not clear to me: when i encode and decide to restart the pc for any reason - i have [enable resuming] checked - how do i actually do the resume? i don't see any button for that?!?


press ctrl-c to pause it then to restart it run it again without the enable resuming switch.

DaveEL

[Beave]

OK, now redownloaded it. It gives the following error messege now:

AVIFileOpen failed to create destination file Error 1 test.avi : Das System kann die angegebene Datei nicht finden.

[DaveEL]

Originally posted by Beave
OK, now redownloaded it. It gives the following error messege now:

AVIFileOpen failed to create destination file Error 1 test.avi : Das System kann die angegebene Datei nicht finden.

Microsoft decided to not document that error! new build up which should return a little more in the error message.

DaveEL

[DaveEL]

Originally posted by Nic
The best solution is to write your own or use another avi output routine (vidomi's code is quite good)...


Been wanting to do this but i havnt found one which does what i need except the avifile ones. The one missing feature is that i need to be able to open up an existing avi and append to the end of it. If i cant solve the problems with the avifile api ive been having ill have to make a copy of the avi and then copy frames back to the original before starting to compress. Also one small problem is one of licensing most of the avi write routines ive found are gpl however no explicit license is on avs2avi yet - Toff and int21h do either of you have a problem with relicensing your code gpl so we can use these routines.

DaveEL

[[Toff]]

Originally posted by DaveEL
Toff and int21h do either of you have a problem with relicensing your code gpl so we can use these routines.

No problem for me.
int21h asked me the same thing some times ago, so I think he doesn't have any problem and would like to relicense the code to GPL.

If someone want to test an alternative to the vfw avi writer, I've made a test with the virtualdub one some times ago :
http://christophe.paris.free.fr/temp/avs2avi113_vdub.zip
it's old so use it as your own risk :D
(also it still use vfw to read the avs file)

[DaveEL]

Originally posted by [Toff]
No problem for me.
int21h asked me the same thing some times ago, so I think he doesn't have any problem and would like to relicense the code to GPL.

If someone want to test an alternative to the vfw avi writer, I've made a test with the virtualdub one some times ago :
http://christophe.paris.free.fr/temp/avs2avi113_vdub.zip
it's old so use it as your own risk :D
(also it still use vfw to read the avs file)

Thanx ill take a look at that try to update it to the latest code and write the resuming support for it.

@Beave can you test this one as well at my latest test build and see if you have any problems.

DaveEL