Subject: Re: CCL:Linux Cluster

From: Avijit Ghosh <avijit)at(physics.drexel.edu>
Date: Mon, 3 Nov 2003 22:42:58 -0500 (EST)

We just bought our cluster from a vendor (Western Scientific).
I would recommend going to a vendor if you can afford it, mostly
to guarantee there won't be any hardware incompatibilities that
are sometimes hard to work out until you start setting everything up.

Hardware Linux vendors will  also quite possibly be more  up to date
on what the best configurations/tools might be. 

Price it  out and take a look at the relative costs of going the "do it 
yourself" route and from a vendor. (In our case, our prices were pretty
competitive w/ buying cheap 1U dual xeon blades from bulk hardware 
companies  like www.mwave.com and the support is absolutely brilliant). 

If it looks like it is much cheaper going the do it yourself route,

take a look at the following: 

http://oscar.openclustergroup.org/tiki-index.php

Oscar will automate installation over a network and make it reasonably
straightforward to install over an arbitrary number of nodes, setup a 
reasonable queueing system,  install mpi(ch), take care of user accounts 
and maintain patches/updates. The documentation  provides some good advice 
on "reasonable" configurations as well.

Read over the "quick  install" to get a sense of  the work involved for 
setting up a cluster on your own. Its quite impressive how straightforward 
this stuff has become. There is no 6 node limit :)

	best regards,
	-avi

Asst. Prof. Avijit Ghosh, Ph.D.

Dept of Physics
Drexel University
Philadelphia, PA 19104

---

From: Anton Feenstra <feenstra.-at-.chem.vu.nl>
Date: Tue, 04 Nov 2003 09:32:48 +0100

> on what the best configurations/tools might be. 

If I may add some (hard-learned) notes about do's and dont's in hardware:
- check linux support for your networkcard
- check for available options in the bootrom of your networkcard (if you're
   going to use that, you probably would want to)
- get a reliable motherboard (don't go for the cheapest)
- oreder the boxes pre-installed with your flavor of linux, that eliminates
   a number of possible (trivial, but annoying and time-consuming) harware
   problems (like loose cable connectors, or memory banks...) Run a memory
   test and cpu-burn when you get the machines. (I've spent months configuring
   and re-configuring my first two nodes, which turned out to be faulty...)
- get boxes with lots of holes and put fans in all of the holes (get the
   expensive ball bearing ones, too, others fail too soon). See to it that
   cables don't obstruct airflow in the box.
- do install a graphics card - wonderful if you need to find strange errors.
- don't get an (expensive) keyboard/graphics switch. Plugging cables also works.
- using dual CPU machines saves you in operating cost - less hardware to
   maintain.
- if floorspace is not an issue, boxes on shelves do just as well as 19"
   rackmounted stuff.

Well, maybe this is all too obvious to most of you, but I've seen people
making, well let's call it 'less well-informed decisions' about buying
cluster hardware, so I thought I'd just add my two-cents...

-- 
Groetjes,

Anton
  _____________ _______________________________________________________
|             |                                                       |
|  _   _  ___,| K. Anton Feenstra                                     |
| / \ / \'| | | Dept. of Pharmacochem. - Vrije Universiteit Amsterdam |
|(   |   )| | | De Boelelaan 1083 - 1081 HV Amsterdam - Netherlands   |
| \_/ \_/ | | | Tel: +31 20 44 47608 - Fax: +31 20 44 47610           |
|             | Feenstra.-at-.chem.vu.nl - www.chem.vu.nl/~feenstra/       |
|             | "If You See Me Getting High, Knock Me Down"           |
|             | (Red Hot Chili Peppers)                               |
|_____________|_______________________________________________________|