Subject: Re: CCL:Linux Cluster

From: Eugen Leitl <eugen..at..leitl.org>
Date: Sat, 1 Nov 2003 19:14:52 +0100

On Fri, Oct 31, 2003 at 10:04:11AM +0100, Kenneth Geisshirt wrote:

> My point is that building a cluster by hand does not scale well. If you 

This is true, but the real issue is power supply and air conditioning.
The limit for this used to lie around 64 nodes, but is now probably lower due
to current increase in power dissipation density (1/4 KW in 1 U for dual Xeons):

	http://www.beowulf.org/pipermail/beowulf/2003-March/006107.html

It is still feasible to dissipate the heat by venting a hall of loosely
shelved desktop PCs into open air -- assuming one doesn't have to settle for
existing, crammed building floorplan -- which is probably the case for the majority of
new installations in established institutions.

> plan a large cluster (32+ nodes) you will need some kind of automated 

All cluster-centric Linux distributions use some kind of install automation. It's not
rocket science, Beowulf list archives are full of references

	http://www.beowulf.org/pipermail/beowulf/

> tool. If you build a 4-node cluster and add a new node (different 
> hardware e.g., an old office pc) you can do it by hand.

Typically, it pays to NOT recycle old _and_ heterogenous hardware. Due to the
current innovation rate any ~k$ office-grade PC node (currently averaging at 
64-bit x86 (Opteron/AMD 64) and GBit Ethernet) will outperform a large assembly of
old recycled systems, even not considering scaling issues (even well-behaved code
tends to profit from comparatively expensive low latency message-passing fabric such as
Myrinet or Infiniband -- latter's large-scale deployment being pioneered by the
Virginia 1100x G5 2GHz cluster).

For small-budget installations 8-16 nodes would still tend to form a sweet spot,
due to recently-fallen prices for 8-16 port SOHO GBit switches with full
backplane bandwidth.

Whether these will actually do it for you still depends on benchmarking your code, or
at least a representative mockup thereof.

-- Eugen* Leitl <a href="http://leitl.org">leitl</a>
______________________________________________________________
ICBM: 48.07078, 11.61144 http://www.leitl.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE