Subject: Berkeley Linux VIA v2.2 released

From: [email-masked]
Date: Mon, 15 Mar 1999 11:51:18 -0800

Caveat: this is a heavily commented bounce. Original message from Matt 
Welsh follows below the From: line. Apologies for possible
technobabble, but this is not just a tangent to CCL.

M-VIA is a fast user-level message-passing library which supports DEC
Tulip Fast Ethernet and PacketEngines' Yellowfin GNIC-I Gigabit Ethernet 
network adaptors. Below announcement hints at forthcoming support for
Myrinet, a very fast yet expensive proprietary cluster networking
solution from Myricom. M-VIA achieves a ~60 us pingpong latency on 
DEC Tulip Fast Ethernet, a $30 network interface card (NIC) to be had 
from many vendors (but beware of NetGear NICs with Lite-on chipset,
the older versions are ok. Kingston is ok). A single GBit Ethernet 
NIC can saturate the PCI bus. There seems to be no sense in using 
more than 4 NICs/node even if the slots and irqs are available, 
as a 300 MHz CPU will be mostly handling the packet traffic then. 
Linux supports up to 4 NICs sans any kernel patching, so for small 
clusters (4..8 nodes) you can omit the switches and employ full 
duplex (i.e. 2 Gbps for GBit Ethernet and 200 MBps for Fast Ethernet) 
over cheap crossover Fast Ethernet cable. With Beowulf kernel patches 
channel bonding up to four channels is possible, so multiply above 
numbers by the number of channels bonded. Current Fast Ethernet 
switches which allow cut-through switching introduce few us latency, 
while offering at least 1 Gbps backbone bandwidth. ASIC-based 
cut-through 8-porters with 1-2 Gbps backplane bandwidth are to be 
had for slightly over $50/port. (Insist on cut-through as the 
switching method, _not_ store-and-forward which means ms latencies 
for large packets).  

Extracted from http://www.nersc.gov/research/FTG/via/faq.html 

Notice: TCP relates to Linux TCP implementation. Your data will be
significantly worse if you use a workstation. I have no idea 
whether below relates to full duplex or not.

Network                  Protocol  Latency (us) Bandwidth (MB/s)
Packet Engines GNIC II   TCP       59           31
Packet Engines GNIC II   M-VIA     19           60
Tulip Fast Ethernet      TCP       65           11.4
Tulip Fast Ethernet      M-VIA     23           11.9

According to the FAQ, an MPI version utilizing M-VIA as transport 
layer is in the works. Notice that according to Beowulf mailing list 
there seems to be a bug in Linux' TCP/IP stack which boosts the 
latency of every 40th-odd packet from few ten us to ms range, so 
this is very relevant for those CCL members who want to set up a 
Beowulf cluster. Also xref my other message to CCL about the pgcc 
compiler. Other possible eye-opener: NAS Serial Benchmark Class 
W Performance page, which states: "The key result is that starting 
with 400 MHz PII processor, performance of Intel performance (sic) 
is comparable to some high-end processors. In particular, it gets 
approximately the same as a T3E-900 processor (450 MHz alpha). 
The T3E-900 is a current generation supercomputer". For more read
http://www.nersc.gov/research/FTG/pcp/performance.html

See also "Avalon: An Alpha/Linux Cluster achieves 10 Gflops for $150
k", a paper submitted for the 1998 Gordon Bell Price/Performance
Prize (available as ps.gz on the web), "Lightweight Computational 
Steering of Very Large Scale Molecular Dynamics Simulations" at
http://bifrost.lanl.gov/~dmb/SWIG/papers/sc96/INDEX.HTM 
and "Feeding a Large-scale Physics Application to Python" at
http://python.via.ecp.fr/workshops/1997-10/proceedings/beazley.html
Another very nice illustration of using Python in scientific
applications is Konrad Hinsen's Molecular Modelling ToolKit MMTK, 
which became the basis for OpenChem, the first fully interactive 
multiplatform chemistry simulator ;)
http://www.openchem.org 

For a summary of scientific codes available for Linux, both OpenSource 
and commercial, see

http://sal.kachinatech.com 

specifically under biology and chemistry.

Another tidbit: according to private communications, the next Alpha family
member will offer very fast interprocessing communication links, which 
would make most of abovementioned networking cruft obsolete. However,
Alphas are notoriously pricey, which is not true for AMD K6 (K6-III
with 256 kByte on-die cache is out, release of K7 is imminent) and
Intel Celeron Mendocino based machines, which seem a better choice for 
memory bandwidth bottlenecked and better-scaling parallel codes.

Comments welcome.

Regards,
Eugene Leitl

>>>>>>>>>>>original messages follows>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
From: Matt Welsh <[email-masked]>

Berkeley Linux VIA v2.2 release, March 15, 1999

Berkeley Linux VIA v2.2 is an implementation of the VIA Virtual Interface 
Architecture for Myrinet LanAI 3.x/4.x boards and Linux 2.2.x.

It is available for download at:
  http://www.cs.berkeley.edu/~mdw/proj/via/berkeley-linux-via-2.2.tar.gz

For more information on the Berkeley VIA Project, see 
  http://www.cs.berkeley.edu/~philipb/via/

New in this release is support for Linux 2.2.x kernels.

Berkeley Linux VIA is a high-performance implementation of the Virtual
Interface Architecture over Myrinet LanAI 4.x boards and Linux 2.2.x.
It provides a round-trip latency of under 70 microseconds for small messages,
and over 410 Megabits/sec bandwidth for large messages (4 Kilobytes).

One of our future goals is to integrate this code with the "M-VIA" 
Modular VIA implementation from Pat Bozeman and Bill Saphir at LBNL. 
We have discussed the plan of attack with them and hope to do this for
a near-future release.

For more information on the Virtual Interface Architecture, see
  http://www.viarch.org

Berkeley Linux VIA is by:
  Matt Welsh <[email-masked]>   
  Philip Buonadonna <[email-masked]>
  Andrew Geweke <[email-masked]>

Please contact us if you have any questions or comments.

The code is covered under the following copyright license:

 Copyright (c) 1998, 1999 by The Regents of the University of California.
 Copyright (c) 1995, 1996 by Cornell University.
 Copyright (c) 1997, 1998, 1999 by Matt Welsh.
 All rights reserved.

 Permission to use, copy, modify, and distribute this software and its
 documentation for any purpose, without fee, and without written agreement is
 hereby granted, provided that the above copyright notice and the following
 two paragraphs appear in all copies of this software.

 IN NO EVENT SHALL THE UNIVERSITY OF CALIFORNIA BE LIABLE TO ANY PARTY FOR
 DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT
 OF THE USE OF THIS SOFTWARE AND ITS DOCUMENTATION, EVEN IF THE UNIVERSITY OF
 CALIFORNIA HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

 THE UNIVERSITY OF CALIFORNIA SPECIFICALLY DISCLAIMS ANY WARRANTIES,
 INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
 AND FITNESS FOR A PARTICULAR PURPOSE.  THE SOFTWARE PROVIDED HEREUNDER IS
 ON AN "AS IS" BASIS, AND THE UNIVERSITY OF CALIFORNIA HAS NO OBLIGATION TO
 PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.

Happy hacking!
The UC Berkeley VIA Team