ACM audio

[dividee]

I've hacked some ACM audio support into AVISource, but it doesn't work properly. At least I can hear something from a few AVIs.

If some developer want to have a look, please do so.
* I'm not sure about calculations of stream length and seeking positions
* When you seek you hear the end of the previous buffer and I can't find the bug
* Don't ever hope for VBR support!
* It probably needs some protection against wrong stream headers
* Don't know how to handle audio skew.

Most of the code is in the following methods of AVISource: LocateAudioCodec, DecompressAudio, NextAudioChunk, AudioSeek and a bit in AVISource (the constructor) and GetAudio.
You have to add msacm32.lib to the library modules for the linker.

By default ACM audio decoding is not activated, as most of the time it barks an error message (and probably crash sometimes too), I added a boolean parameter to AVISource (& variants) to activate it:
AVISource("test.avi",true)

There is no binary in the attachment, because as I said it doesn't work properly (yet).

[Richard Berg]

VDub's AVI reader is perhaps its best feature -- no bugs I'm aware of, and does a great job compensating for corrupted files & bugs in other programs. We already use a lot of VDub code in this department; perhaps all we're lacking is a sync-up with the latest from Avery?

[dividee]

AFAIK, none of vdub audio code is in avisynth. I've written that with an eye on vdub sources, but maybe you're right, I'll look into a more direct integration. It's just that vdub sources are a little above my skills and at time a bit difficult to follow.

[dividee]

I've redone it using a different approach. I sync'ed virtualdub files with the current release of vdub and imported audio-related files. Adding audio support after that only involved about 10 lines of code, and adding a seek method in AudioStreamSource (it was only coded to allow streaming and skipping samples, not for random access. I hope i got it right).
All in all, it sums up to a ~50K bigger release dll.

Anyway it seems better now, most notably with files with wrong headers.

This time I attach a binary. There is still an optional boolean parameter to AVISource & co. to activate audio, but this time it defaults to true as I think it works better.

Note that you can also open standalone mp3 files with it, if you 'wavify' them before.

I'll wait a bit and if there are no complaints i'll add the code to the CVS.

[Richard Berg]

Great job! Haven't tested it thoroughly but it seems like there's a lot of pent-up demand that'll do it for us :)

I suppose you suggested the next immediate feature: being able to add RIFF headers to arbitrary files. I can't find source for WavMp3, but if anyone knows where something similar can be found (BeSweet perhaps?) let me know & I'll hack in a RiffSource() function.

[Koepi]

The nandub sources should contain such a routine for the VBR mp3 stuff.
Too bad i don't have the sources anymore, but hopefully at doom9's download section you should find them.

Regards,
Koepi

[Richard Berg]

I'd recommend staying away from Nando's audio-interleaving code. That's a recipe for breakage (http://virtualdub.org/virtualdub_news_old.html).

[Koepi]

Erm, the stuff for adding riff-headers could still be useful, don't you think?

I know of the problems avi has with vbr mp3, guess why I do that OGM stuff ;)

Regards,
Koepi

[dividee]

Hehe thanks for the link, Richard.
I should have read this before starting. In my initial attempt, I was using a nandub avi+mp3 (do someone remembers the SBC intro ;) ) for debugging and scratched my head when I saw that nBlockAlign was the number of samples in an MP3 frame (why??).
I ended up with code that read this file correctly (even seeking worked) but didn't work with anything else. Then I changed it to correctly handles other files, but it broke Nandub files. That's the code I posted in my initial post above.
Now, with Avery code doing most of the work, it (seems to) work for both regular and Nandub files. I guess he implemented the work-around he talks about in this article.

[trbarry]

Dividee -

Very cool.

But I don't know what ACM stands for. What types of audio can it now read? MP3? AC3?

If it could read ac3 somehow then we could save the name of the audio file in the d2v file and have mpeg2dec make it available, say to an MPEG2AC3Source filter. Doable?

- Tom

[Koepi]

ACM is the windows pendant to VFW/WDM/DShow/DirectX for audio - the Audio Compression Manager (ok, the samples are a little too general ;) ).

It does the same as DShow does with the FourCC, it selects the right decompression codec AFAIK.

Regards,
koepi

[dividee]

Virtualdub uses ACM to open audio streams in AVIs. It can read whatever audio codec you can see in the 'Sound and Multimedia' control panel under 'Audio codecs'. Unfortunately, I don't know about any AC3 or vorbis ACM codec, only dshow filters (maybe I'm wrong?). So it's mainly useful for mp3.

[Koepi]

vorbis acm can be found here:

http://hp.vector.co.jp/authors/VA012897/

(sorry, site is japanese ;) but you can read the links, they're western fonts).

For ac3... well, my kx-drivers ( http://www.kxproject.spb.ru/ ) can do that, dunno for an real ACM filter though, too.

Regards,
Koepi

[wing1]

@dividee

Thanks for the added support for compress audio. It seems to work with virtualdub compressed mp3 audio. I tested on a sample clip which I captured with Xvid and compressed audio to mp3.

The script I used :

Import("c:\avisynth\plugin.avs")
vidin=OpenDMLSource("e:\sample.avi")
vid1=vidin.tweak(bright=23,cont=0.92)
vid2=vid1.Cnr2(mode=-1,ln=25).MergeLuma(vid1.TemporalSmoother(2,1).unfilter(0,55))
vid3=vid2.ConvertFPS(23.976,zone=48,vbi=51)
vid4=vid3.BicubicResize(640,240,-0.5,0.75)
vid5=vid4.TomsMoComp(1,-1,1)
movie=audiodub(vid5,vidin) #<------This is optional..seems to speed up the frame server a little for some reason...
return movie # use vid5 if audiodub is not used

The audio came out as 44.1Khz 16bit Stereo PCM

[trbarry]

Would this AC32WAV code (http://mukoli.free.fr/ac32wav) be of any use in Avisynth support for AC3?

- Tom

[Marc FD]

Ah enfin ! t'as assuré sur ce coup la ;)

Great job dividee ! I'll wait for the next avisynth binaries on sourceforge... I eager to use Cnr2 to postprocess some bad encoded movies (not by me of course :D )

[sh0dan]

dividee, this truly rocks! :cool:

I actually also had a look at it myself this weekend, but I couldn't get anything working.

I'd say, as long as it can be disabled (using the bool), it seems safe to commit it.

While we're on the subject I think implementing floating point samples would be a great thing for Avisynth. For starters I think a simple addition of a boolean to the VideoInfo struct and some converters back and forth could do the job. But I don't know about compatibility, and I wouldn't risk breaking anything.
I myself wouldn't mind adding float-point mode to the existing filters, and I don't know of any existing filters.

A suggestion for a more radical implementation could be:

Replacing:

int audio_samples_per_second; // 0 means no audio
int num_audio_samples;
bool stereo, sixteen_bit;


With something like:

int audio_samples_per_second; // 0 means no audio
int num_audio_samples;
int bytes_per_sample; //Not multiplied by channels, (ie. 8,16 when using ints, 32 when using floats)
int channels; // Does it make sense to have more than 2 channels, since exporting them would be hard?


If >2 channels doesn't make sense, then the 'bool float_samples;' is probably the safest to use.

[Richard Berg]

I don't think multiple channels do much for us. Support for more than one audio track, OTOH, would be awesome -- too bad VDub doesn't do this.

As far as floating-point goes, I think it's a logical step forward. The performance hit will be small since audio just ain't that much information compared with video, and the extra precision is more necessary with audio than anything else especially when you start talking about resampling (ears are very sensitive, not to mention audio encoders...)

The best news is that there isn't a huge body of existing audio filters to change like there would be when we were talking about messing with the video formats.

[sh0dan]

Would it be safe to add "bool float_samples" to VideoInfo? (this _is_ a basic question I know, but I don't want to mess anything up).

[Richard Berg]

Wouldn't hurt, but I like the bytes_per_sample idea better. ("Int channels" is a definite improvement too, even if we don't find a use for channels>2 until version 50). You can add it without worry, just don't remove the two bools until we've updated the rest of the code.

[dividee]

I hope there are no external audio plugins. If we change the interface, we better do it right the first time.
If we would ever want to make such kind of changes to the video information, it would break compatibility with most plugins.

[trbarry]

Pardon again my ignorance of audio, but it would be nice to stub in the possibility of more channels, even if not much could currently be done with them. It seems most everything is (slowly) going to multi-channel audio and with DVD-R's down to a buck apiece the cost of saving the audio is often unimportant.

So we might not be bothering to shrink audio as much in the future once better support arrives in associated utils.

- Tom

[Richard Berg]

Agreed -- for now AC3 and DTS are much more practical, but future expandability is painless.

[poptones]

Does setting "sample rate = 0" turn off ALL audio processing? If you're going to add a bunch more processing code, don't forget to turn it off when we don't need it. I'm asking (of course) because I will almost never need it, and I'd like to turn it off to keep up encode speed.

Also keep in mind future audio standards are almost sure to run to 192K/24 bits.

[sh0dan]

Only filters processing audio will have to actually care about audio, it just passes straight through most filters. Otherwise I just added killaudio(), that makes them not care :)

192K doesn't matter much, since we deal with uncompressed audio all the time (when it has been imported), and using floating point is more than enough precision for 24bit/sample.

Implementing 'int nchannels' instead of 'bool isstereo' is probably the way to go, just in case. Does ANYONE know ANY external audio plugins for AVS?

@dividee: How good do you feel about your compressed audio - is it ready for 'prime time'? :)

[dividee]

Out of curiosity, I tested Edwin's AVISourceEx with following results:
AVI+MP3(CBR): linear playback ok, bad synch on seek.
AVI+MP3(VBR): "audio could not be read from file"

With the vdub code I imported in avisynth:
AVI+MP3(CBR): linear playback ok, synch on seek ok (AFAICT)
AVI+MP3(VBR): linear playback ok, bad synch on seek.
Actually I got the same behaviour in vdub 1.4.10, predictably.

I found a bug: when you open a RIFF file with an unhandled audio stream, you correctly obtain an error message, as the code from vdub throws an avisynth exception (error.h took care of that most of the time and I modified vdub sources in a couple of places) but the application then crash when you close it.
Unfortunately I can't do anything about it now as my monitor died and I'm stuck with a laptop which doesn't have VS installed (and I can't install it). I'll have to shell out some €€€ to get a new 19' or... well I don't have any choice :(
So I'll post the relevant sources as they are now but be careful when merging!
* I include my .dsp as it needs some new libraries for linking
* All files should be put in 'Virtualdub files' except (of course) source.cpp and clip-info.h (which was a part of source.cpp I splitted because some VD file needed these defintions).
* Note that you can get rid of FastReadStream.{h,cpp} since they are not used.

It would be nice if someone could squash the aforementionned bug before merging in the CVS :devil:

I should have done this myself, but I probably won't be able to use my main computer until the week-end (I'm desperate).

[Marc FD]

it's a dream becoming true :)

Any ideas for ac3 import trough MPEG2source ??
it's not as important (for me) but i think it could be a GREAT feature :cool:

(i'm happy, thanks too dividee i'll never think :
"i could do such great things with AviSynthEx, but i hate it !"
i'll just ignore it :) )

[Richard Berg]

I've got the beginnings of a RIFF writer working at home to replace the (closed-source) WAVMP3. If I find the appropriate docs it'll be very simple to adapt it for AC3, OGG, whatever.

Yes, I could've looked at the Nandub sources. No, it wouldn't be as fun as working from scratch :)

[dividee]

Not much feedback on the dll I posted. Should I assume it works correctly ? Anyway, I managed to install VS.NET on my laptop (don't have VS6 discs handy) and I'll commit the code to the CVS. I'll commit the project file I had from VS6 (but I can't double-check it; if it cause any problem go back to the previous version and add msacm32.lib and winmm.lib to the library modules for all configurations) as well as the VS7 one. BTW I forgot some files in the zip above. Sorry to anyone that tried to compile it.

[Richard Berg]

The DSP references tempsmooth.cpp/h but the files aren't on the CVS.

[dividee]

Arg! Could you dump these references, please ? I don't feel like hand-editing a DSP file :p

[Richard Berg]

Ok, didn't know if you were planning to integrate TemporalSmoother.

[dividee]

Actually I don't plan to integrate it anymore, but I never removed the files from my source directory nor from the DSP

[sh0dan]

@dividee: I got the code compiled, when you posted the source here, but I couldn't reproduce the error (haven't got any bad AVI's) - so actually everything worked fine.

[sh0dan]

Just uploaded a binary for testing at the usual test binary place (http://cultact-server.novi.dk/kpo/avisynth/avs_cvs.html). Now everybody: Test! Test! Test!

[Marc FD]

aye aye, sir !

[Richard Berg]

@dividee -- I also removed references to scenechange.cpp/h, let me know if this was in error.

[dividee]

Just found a bug with the new audio code:
You can't open two MP3 streams in the same script:
AVISource("f1.avi")
AVISource("f2.avi")
gives an error when opening the second file if both contains MP3.
It works if one file has MP3 and the other has PCM or GSM encoding, for instance.
PCM audio doesn't use ACM for decompression, but GSM does.
Tried GSM+GSM and it works. Seems to be a problem with the Fraunhofer codec ?

[western shinma]

I haven't had any problems with opening several files with mp3 audio in a script myself, but I didn't try the old radium cracked version.

[dividee]

Since I installed divx3.11, I think it installs the radium codec too. I'll try with another one.

[edit:] I installed the standard codec, but it doesn't work. Might be some peculiarity in my system. Well, as long as it doesn't affect anyone except me, I don't care.

[spyder]

There is already a GPL MPA2WAV(Adds riff headers to mpeg audio files) and AC32WAV(Adds riff headers to AC3 files) at http://mukoli.free.fr

robUx4 has completed these classes as a starting point for integrating support for these files into VirtualDub and making MPA2MCF tools etc.

EDIT: I didn't mention that he also has command line tools that use his C++ classes to add the headers.

[WarpEnterprises]

@dividee: About the new parameter "audio" in the Avisources:
Is it's meaning:

If TRUE then use ACM to extract audio,
else assume the audio is PCM and use the (Avisynth built in) decoder

(For adding to the docu)

[dividee]

No:
true (default) = audio on
false = audio off

if the audio is PCM it never uses a codec to decompress it, but setting "audio" to false still deactivates it.