network cpu's together to encode faster

[pkiley]

I know this gots to be possible, but I cant figure out how to do it. i found research products that give out clients with setting tweaked to help you give them your cpu cycles over the net so they can find cures to cancer and stuff. Although I want to do that too, I would really like to combine my home computers together so I can do quicker encodes. I like to use gknot, but I'm up for any options for xvid encodes. I'm not sure what sound to use, but for hardware compatablility i think i may use AAC. I much rather use MPC, audio monkey, or ogg but not sure if I'll regret it later if i buy a hardware divx/xvid home player. I would like to use .mkv as the shell as well. Thanks for anyhelp you can give.

While I'm askin questions does anyone know where to refer me to a discussion about adding deleted scenes on a seperate video stream within a .mkv file. I don't want to put them at the end of a movie because I want to option from the get go, but if at the end is the best way to do it, then so be it.

Peace
idi

[Selur]

www.vidomi.com got network encoding support, a older version of VirtualDubMod also supported this,.. so some search in the board might help ;)

Cu Selur

[Turbonutter]

VDM's network encoding never really worked.

[superdump]

...which is why it was removed.

[b0b0b0b]

This is a trivial solution, but can you do one movie per machine?

Still pretty trivial, but what about networking a drive, and then doing a pipeline of movies? One machine could be the dvd2avi machine, one could do one pass, another could do another pass, another could do audio encoding, ....

[pkiley]

ideally the only thing suitable for me is called distributed computing. like in this program used to develope cures and treatments to diseases. They can have millions of cpu's hook up if they want and its really easy. I just want to hook it up to my virtualmoddub instead of their program. share cpu cyles over the net or network. How great can that be. If like 10 of us have the same dvd we can all encode it together and get done in like an hour and all of us would have our own copy. Can you imagine the possiblilities. And when we are done, give cpu to cancer or whatever you want.

This is a developeres tool to incoperate it into your app. So if someone thats got programming skills can easily put it into virtual dub for testing purposes. I hope somebody takes this on
http://www.gridironsoftware.com/product/overview.asp


http://www.aspenleaf.com/distributed/distrib-2000/distrib-news.html
some others explaining the process or their particular project
http://www.extremetech.com/article2/0,3973,288866,00.asp
http://news.zdnet.co.uk/hardware/chips/0,39020354,2085459,00.htm
http://www.dvd.reviewer.co.uk/info/fightcancer.asp
http://www.iceteks.com/ud.htm

[Prettz]

I'm not understanding how this could work. Doesn't the codec need lightning fast access to huge amounts of data (video and otherwise) during the entire process, and need to know about the predicted bitrate for the entire movie in order to make decisions? It's bad enough with long memory access times, but how could doing this over a network possibly make things better?


edit: distributed computing only works when the problem can be split up into several completely independent tasks, all of which are extremely computationally intensive, not memory-access intensive.

[Danzel]

Although im not an expert on either distributed computing or Xvids internal working, I think that Xvid encoding could be distributed (over a home network).

The first pass of Xvid is basically encoding the whole movie at a constant quant 2 to see how big each frame comes out.
Each computer could process a portion of the stream and save its logfile, then there is already a tool for joining logfiles together, although it doesnt give optimum results. (i think keopi released it?)

The second pass could possibly then be done by having a master system scale down the logs and hand out portions to be encoded to all the machines, to encode and then be joined together by the master.

Anyone got any views on this?

Danzel.

[doug_s]

There's an app written for linux called dvd::rip that does exactly what you're looking for. I've only used it on one computer but other linux users have reported that clustering mode works pretty well.

Here's the link:

http://www.exit1.org/dvdrip/doc/cluster.cipp

It's an open source project so if you don't want to use linux you could still download the source code and see how it works

[Didée]

This topic was already discussed extensively, though I actually don't remember the thread and am too lazy too search ;)

Danzel:
The idea is clear, but wont't give optimal results. What's with ... overflow treatment?
Basically, your idea would be delivering the same result like:

<assume you have 10 PCs available>
- machine 1 does 1st-pass on [0%-10%[ of the movie
- machine 2 does 1st-pass on [10%-20%[ of the movvie
...
scale down each *NN.stats by adding them, scale down, divide *NN.stats by ( CompleteStats / CompleteStats_scaled )
...
- machine 1 does 2nd-pass with *01_scaled.stats
...

Harder to describe than to do ;)
You already can do that manually, with a calculator, it's 5 minutes of work. Not automated, yes ... but fast in the end.

- Didée