network encoding with clusterKnoppix?

[avih]

i just had an idea.
how about using clusterKnoppix ( http://bofh.be/clusterknoppix/ ) for encoding? (mencoder, vdub or others via wine, etc).

i'm not a linux/mosix guru, but isn't openMosix on the kernel level? meanning that any task is shared over the network?

if so, knoppixCluster is fully automatic. apperantly, the cd is needed only on one machine, and the others can boot via network, and talk among themselves and balance load automatically.

could be really nice if it's working. i.e. one can stay at the office after ppl leave, turn off few machines, insert clusterKnoppix into one of them, boot the machines, and voila, cluster ready for some heavy duty tasks.

do u think it's possible?

[Belgabor]

I think the cruicial point here is multi-threading. If the encoding app can multi-thread it should work ok. But depending on what you do you might run ito bottlenecks, read speed, network throughput etc.

[jggimi]

Well, I may give it a try. As Belgabor states, it will depend on the number of processes used in the encoder (for parallelism), as well as where the source and destination files are located on the network (for I/O throughput). The latter may have a siginificant impact with this version of the OS, since the slave nodes are apparently diskless.

In addition, the capability of the individual CPUs in the cluster may have an impact; for example, if I'm encoding with a P4-optimized process it may or may not run properly on an Athlon.

[CaptainCarrot]

If you try this I'd go for a single CD for every PC you use, first to decrease network traffic, and second because you can change all PC's first boot option to CD, and they still boot their normal os if you just take out the CD. Whereas if you're running your master and some other persons want to do some work on their pc configured for first boot option LAN they'll go nuts because this freakin knoppix will boot all the time and they don't know what to do about it... :D

[avih]

Originally posted by CaptainCarrot
If you try this I'd go for a single CD for every PC you use, first to decrease network traffic, and second because you can change all PC's first boot option to CD, and they still boot their normal os if you just take out the CD. Whereas if you're running your master and some other persons want to do some work on their pc configured for first boot option LAN they'll go nuts because this freakin knoppix will boot all the time and they don't know what to do about it... :D

so it's about time they learn ;)

[redcane]

I doubt mosix is optimised too well for this...

If the processes aren't split well between the machines, they will be blowing *heaps* of data across the network, which will be a big bottleneck.

Transcode has a built in network transcoding system, which is probably more efficient because it breaks the data up into efficient blocks to be sent to different machines. There are also boot cds around the net that boot a system into a cluster transcoding box. They've used this at my office, and it works really well.

[ross_MoHaX]

i just had an idea.
how about using clusterKnoppix ( http://bofh.be/clusterknoppix/ ) for encoding? (mencoder, vdub or others via wine, etc).

i'm not a linux/mosix guru, but isn't openMosix on the kernel level? meanning that any task is shared over the network?

if so, knoppixCluster is fully automatic. apperantly, the cd is needed only on one machine, and the others can boot via network, and talk among themselves and balance load automatically.

could be really nice if it's working. i.e. one can stay at the office after ppl leave, turn off few machines, insert clusterKnoppix into one of them, boot the machines, and voila, cluster ready for some heavy duty tasks.

do u think it's possible?

What was your expirience? Does openMosix suitable for this task?