plugin that change images based on audio rhythm

[kobayashi]

Hi. Does anyone knows about a plugin that from a set of images and an audio stream can create a video where frames are the images of the set changed at rhythm of the audio stream?

It should be interesting also change the rapidity of changing of frames of a video stream at rhythm of the audio file.

Imagine a video made from a set of a rock band concert photos changing at the rhythm of the music of the band...

[E-Male]

that would need an algorythm detecting "rhythm"

what you could do is that:
use a bpmcounter (freeware avaible) to get the tempo
then use imagesequence, and fps-conversion commands to sync

if that way is ok with you i'll go into details

[kobayashi]

Originally posted by E-Male
that would need an algorythm detecting "rhythm"
use a bpmcounter (freeware avaible) to get the tempo

Interesting way... The problem is that it should be more dynamic. The tempo may change. I should divide each part of the song in which the tempo doesn't change...

I have just seen a video that I think has been made with am automatic tool that I dont know.

[E-Male]

i highly daubt you'll get that fully outomated in avisynth
-someone would have to either write or port and modify beat detection code, seem difficult to me, but i've not worked with audio code yet
-i'm not sure if this is at all possible in avisynth, i guess it is, but as i said much work

you'll have to do it by hand, as i described
as i said, if you are interested i can make you a guide, should onyl be a few lines anyway
and i expected tempo changes, you have to segment it for that

[Zarxrax]

What I would really like to see, is similar to this, an avisynth plugin that will let you display winamp visualizations. I often like to be able to capture visualizations to a video stream, but screen capture programs are less than ideal, with lots of dropped frames and such.

[E-Male]

interesting
that should be possible to some extend (the randomnes will be a big problem)
you just need a willing and able programmer

[kobayashi]

I have found something interesting about synching images to audio rhythm:
http://www.joern-thiemann.de/subpage/index.htm?/tools/ssmm/index.htm

From the site:
Slide Show Movie Maker (SSMM). A tool for create slides shows with fading effects, audio, text, and many more...

from FAQ:
9. What does Audio / Video synchronisation mean?

With SSMM, you are able to synchronize audio and video. You hear the audio playing and you are able to select the point of time, when the video changes from one picture to the next picture, so that the audio and the video will be synchronized. For example, you want to switch the picture by the rythm of the audio: just use Audio and Video Synchronisation of SSMM.

I would preferred an automatic process like visualisation plugin of audio players that recognize the tempo, but better than nothing...

[E-Male]

you just expect too much

you (like many other people) got no idea how hard to program some things are

[kobayashi]

Originally posted by E-Male
you just expect too much

you (like many other people) got no idea how hard to program some things are

"I would preferred..." I said, not "I would expected..." ;)

"I would expect that exist a free Operating System better than Windows" is a stronger "hope" than the one of my previous post, but there are more than one free OS better than Windows, in my opinion.
"I would like a good free Image Manipulation Program"... gimp.org

I don't know much about audio, so, in my ignorance I (probably wrongly) thought that picks of an audio stream could be used as an "event" for a changing an image. (sorry for my english!)

[Mug Funky]

beat detection isn't an intractible problem like some other ones we take on (perfect deinterlacing, perfect denoising, recovering detail that isn't there, etc).

fizick already has several plugins that make use of FFT, but that's probably not even needed in this case.

you might be able to abuse the superEQ internal function in avisynth, isolate the low bass (for the kick drums), and then all you need is a peak detector plugin and some fancy conditional code. avs can do random numbers for switching images on beats.

porting winamp plugs would be interesting. i think the main problem would be grabbing the stuff that is GPU accelerated (but there's already people working on GPU integration into a plugin as well).

[edit]

unfortunately i'm not a programmer. because i'd like to see some automated veejay stuff, and i believe avisynth can handle it easily with sufficient CPU thrown at it. the only thing i'd miss is realtime video processing, so the veejay could wave a camera around and it could come out processed (like in that horrible movie virtuosity :))

[E-Male]

@kobayashi:
i didn't mean to offend you, maybe it got out wrong (english isn't my native language, and even in german i often sound rude when i post i na hurry), so: sorry
what meant is that for people not experienced in programming it is hard to tell what is easy and what is hard to program
the rule of thumb is IMO: computer undertsand math, nothign else, you can program what you can convert completely(!) into math

for this special case, the solution you mentioned might really be easier then an outomated one, because it'll give you a result matching your feeling what the "rhythm" is, not the one of some programer


@Mug Funky:
isolating kick drums would work well on trance music, which additionally has a constant tempo mostly (except in mixes)
but rock (play live) seems like a more difficult thing to handle


i must admit, if i was a more experienced programer this challenge would be very interesting, but for now it's by far out of my league


about recoding winamp-visualisation-plugins:
i think the problem would be that teh files in a player just play through
while avisynth is more like gettign random frames at any time
the better soltuion could be a directshow-filter that takes audio as input, a winamp-plugin as option and gives the video as output
this could be used with an mp3 decoder and a video encoder in graphedit

[rfmmars]

This can be all done with MEP2004/2005 from www.magix.com

http://www.amazon.com/exec/obidos/ASIN/B0000AFX5Q/104-8085800-7290348

richard
photorecall.net

[E-Male]

looked through the site, didn't find that feature

[HighInBC]

I tried to do something similiar where the speed of a time lapse would change based on the beat of the music.

I find most beat detection software is lacking, I made a perl script(don't have it anymore) where I hit spacebar every beat manually and it recorded the timing. then another perl scrip(don't have it either) took those timecodes and wrote an avisynth script...

it is alot easier than it sounds if you know perl. Using perl to write avisynth scripts allows for amazing results.

[rfmmars]

E-Male...........yes it does what you want, you can adjuct the video to the BPM and reset the temp also. Magix is the leading music software maker in Europe.

richard

[Leak]

Originally posted by rfmmars
E-Male...........yes it does what you want, you can adjuct the video to the BPM and reset the temp also. Magix is the leading music software maker in Europe.

I dunno - there's also Steinberg, Native Instruments, Ableton and a few more in Europe; and besides one or two pro programs all Magix makes is rather dumbed down entry-level audio software... :/

np: Zorn - Kopfleuchten (The City's Collapsing (But Not Tonight))

[E-Male]

IMO magix is a tiny little bit above kids-software
but they are getting better as a hear
haven't used anything from them for quite some time
i'll think some friends of mine do, maybe i can find out if that software does what is discussed here

[rfmmars]

E-Male.......... if you mean kids stuff like Premire & Vegas, yes you are right.

richard

[HighInBC]

Indeed, adults use avisynth.

I have put together entire features with only avisynth. Simple things like instructional videos. but I could do something harder too, certainly anythings Vegas can do(except those really lame transitions), I use it only to transfer to and from tape.

I do kinda like the 'Film Effect' on Vegas tho, I think trailer park boys used it on their intro.

[E-Male]

Originally posted by rfmmars
E-Male.......... if you mean kids stuff like Premire & Vegas, yes you are right.

richard

now you'Re getting rediculous

what remember magix for is their "music maker"

[rfmmars]

For Leak & E-Male...........it is too bad that your hear says carries any weight at all, for I am sure you have never use the program one time. The program comes in two versions, the MEP2005/005 16 track and the German version VD2+ which has 32 any kind tracks.

With both versions you can multi-frame serve from Avisynth / Virtualdub(xxx) to as many clips you want at any given time, lets see Vegas & Premire do that, and you can modulate any clip or object with audio a million different ways.

I have been a user of these products for three years and know what I am talking about.

What takes you minuits or hours to do some processes in Avisynth takes seconds to do in MEP2004/2005.

I have made 2 scrren shots of how this works but at this time unable to post for some unseen reason. If you will send me an e-mail, I will attach. Use photorecall.net for e-mailing.

Here is a link that is working, it gives you a very small glimps into the product. A .PDF 196 page manual comes with the product which is devoted mostly to making music videos.

www.photorecall.net/Frame1.jpg

By the way, My Vegas 4 and Premire are deinstalled and not used, I can't make money with those programs.

Unless you know a thing is factual and true, don't post 'hear say' You do a poor service to the original member's posting and you were basicly calling me a liar.

richard
photorecall.net

[Leak]

Originally posted by rfmmars
Unless you know a thing is factual and true, don't post 'hear say' You do a poor service to the original member's posting and you were basicly calling me a liar.

Magix brought it upon themselves by making crap like "Music Maker" and *blech* "Ringtone Maker", which is all you see in the average electronic store round here. Maybe they also make usable products, but they really succeeded in hiding that fact behind boatloads of crappy software, as opposed to Steinberg and Native Instruments - which I do own software from.

Not that I want to stop you from using their products, but since you were talking about music software (as opposed to video editing software, which I have no use for, no interest in and largely no knowledge of) it's just not true that Magix would be the best European company in the market.

np: Zorn - Rock The Boat (The City's Collapsing (But Not Tonight))

[E-Male]

well said, magix has a bad reputation and does nothing about it

as i said i'll see if someone i know has this program and test it if possible
i won't buy anything from magix on good luck

also what you could do is post a guide on how to sync pcitures to rhythm, as this is the original question here
maybe kobayashi will buy the program if the guide shows that it's at his skill-level and worth the money

[rfmmars]

Again both of you are wrong..........Mep2004/2005 & VD2+ are Video editing programs NLE, not audio....you do not know what your talking about at all.

All you are is a source of disinformation and a negitive factor to other members who are looking for help.

E-mail me from Photorecall.net and I will attach the 196 page PDF manual if you can handle that large file.

It's too damm bad you can't add anything positive, just your second & third hand incorrect information.

It is your loss, not mine

richard

Photorecall.net

[E-Male]

Originally posted by rfmmars
...It's too damm bad you can't add anything positive...
Originally posted by E-Male
...also what you could do is post a guide on how to sync pcitures to rhythm, as this is the original question here...

[rfmmars]

I made you an offer to attach by e-mail twice the manual and screen captures on how it is done but have I received an e-mail from you?..........NO!

Who in the hell do you think you are? You don't want to know the truth, that is obvious. If I did post, how would I know that you read it? That's the reason for the e-mail attachment.

I have wasted my time dealing with you know nothings.

I will not respond further.

Hope I have been some help to others.

richard

[E-Male]

it's not me that asked for help here, why should i want that manual or screen caps?

i think i am someone that tries to help here

i think you are someone that is more trying to proove that he is right about his taste in programs than about helping the thread starter

i haven't seen anything helpfull from you in this thread! really nothing at all!!

oh, and that screenshot you posted didn't show anything special, just a timeline with some pictures as frames in it

also a free program was posted before, so why add some commercial one...

i start wondering if you work for magix


EDIT1: also, doesn't sending a not freely avaible manual to someone violate rule 6


EDIT2: to make it short either post the how-to for autmatic-rhythm-syncing in MEP or something else that is on topic or if you can do neither of that at least write nothing

[Leak]

Originally posted by rfmmars
Again both of you are wrong..........Mep2004/2005 & VD2+ are Video editing programs NLE, not audio....you do not know what your talking about at all.

Nor do you seem to have much of a long-term memory - this is exactly what you wrote a few posts up:

Magix is the leading music software maker in Europe.

And that was all I was refuting.

All you are is a source of disinformation and a negitive factor to other members who are looking for help.[...]

It's too damm bad you can't add anything positive, just your second & third hand incorrect information.

Dude, all I was talking about is their audio software which is the most visible segment of their product lines around here, and they're doing a really piss-poor job of promoting their less dumbed-down products if it's like that.

And that's all I'm going to say about this matter in this thread, lest you'll again accuse me of spreading disinformation...

[E-Male]

ok, i downloaded MVD2k5 and read the readme:

the feature in question is only avaible in the plus version

it detects the bpm automatically, then you tell it the first beat and to some finetuning

it needs a clear beat

it detects one constant bpm, on tempo changes you need to segment the proccess

so no automatic handling of a rock concert!!

bad luck rfmmars aka mr. magix


@kobayashi: stay with SSMM
it'll give you the output you want, for free, with a very acceptable effort