Subject: Disk solutions in heterogeneous environments

From: John Shelley <[email-masked]>
Date: Tue, 30 Apr 2002 21:14:37 -0700

Dear CCL,
    I'm curious about a cheap relatively modest disk solution (100-500
GBytes) that can be effectively cross mounted on a wide variety of unix
systems (Linux, SUN, AIX, HP, Compaq TRU64 and SGI IRIX) using NFS for
scientific data storage/access.  It need not be a RAID system but could
be. 

   The main reason that I'm asking about this is that I've heard about
severe problems with access to the disk being frequently frozen with
simultaneously cross mounting relatively cheaps disks in
heterogeneous environments using NFS.   

   I'd appreciate any success/problem stories that people might have
regarding this issue.

    John Shelley
    Senior Scientist
    Schrodinger, Inc.

---

From: Szilveszter Juhos <[email-masked]>
Date: Wed, 1 May 2002 08:53:10 +0000 (GMT)

On Tue, 30 Apr 2002, John Shelley wrote:
> systems (Linux, SUN, AIX, HP, Compaq TRU64 and SGI IRIX) using NFS

For a Linux/BSD based solution go for hardware RAID(5) with a reliable
card ( http://linas.org/linux/raid.html ). You can use NFS automounter
(for Linux sometimes better to use the BSD-based daemon instead of the
kernel automounter). Also NFS 3 client/server is supported for Linux
nowadays. If you are using many inodes/files, it is adviced to increase
/proc/sys/fs/file-max (2.4 kernel and also the inode-max on 2.2. kernels)
to its double/triple value. Also better to use the vm33 patch on a machine
that is used as an application server (though it is always a bad idea to
give the fileserver into the users hand ;) Do not forget that for
automounted volumes you have to explicitly 'cd' into the dir, it is not
enough to use the auto-completion in the shell (the TAB key I mean).

Adviced to use a journalling filesystem BUT NOT reiserfs on such systems.
ext3/xfs is the option, I do not know how IBM's jfs is performing, but
that also can be an option.

Backing up such data is a pain if you do not want to pay for expensive
tape robots, etc. People say amanda (GNU) is OK, I have very good
experiences with arkeia (that is not free for this size unfortunatelly but
still many times cheaper than Veritas or HSM solutions). Also due to the
frequent RPC vulnerabilities better to have at least a packet filter
firewall.

So there is no out-of-the box Linux solution, but with some tuning you can
have a pretty good system. I was managing ~2TB on-line data with similar
architecture in a heterogenous net (Linux,HP,Solaris,Windows,IRIX). The
main source of problems was cr@py hardware RAID cards and out-of-the-box
kernels.

Hope it helps:

Szilva