Subject: parallelization & benchmarks

From: [email-masked] (Bill Ross )
Date: Tue, 27 Sep 1994 13:58:39 -0700

I think my 'speed record' could use some more analysis to put
it into context in the real world where everyday jobs are run.
I also describe some improvements in the next version of the
benchmarks.

I believe that Cray's relative lack of emphasis on shared memory 
parallelization is not a major handicap: have you ever seen a Cray 
with fewer jobs than processors? Given the necessary inefficiency 
of running parallel, a job-loaded machine is better off running 
serially, as long as it can fit Ncpu jobs into memory at once 
(Crays don't use virtual memory so must fit the whole job into 
memory). But if there's even one job that's big enough to keep the 
loaded jobs < Ncpu, then parallel begins to make sense. So on the 
typical loaded shared memory machine, one would have to examine the 
memory mix. I notice that an Amber job w/ 10K atoms takes 0.2% of 
the memory on a 1GB Convex, so in the real world I would put shared 
memory parallel of Amber on a Cray down as a frill unless there are 
more idle Crays than I expect, although I have heard of one large
Cray shop where it might be useful. An SGI is somewhat less likely 
to be loaded with multiple jobs, in my experience, which in 
combination with cheaper processors provides more of an incentive 
to parallelize, and large quantum calculations would probably also 
be worth parallelizing on Crays.

By the same token, although we are working on message-passing
parallelism, we are not expecting to use it on our HP cluster, 
since there is usually at least 1 job on each machine and thus 
parallelization would reduce throughput (we will, however, use
it on Paragon, SP2 and T3D machines). We'd need a sophisticated 
batch scheduler to balance onto & off of machines so that a 
process could 'go parallel' when a machine became idle. In fact,
if anyone knows about parallel job schedulers, I would be quite
interested in hearing from them.  Presumably this load-balancing
issue would apply to shared memory scheduling, though there the
operating system kernel might handle it. Richard Walsh has just
informed me that the 'Linda' people claim to have taken care of
the load balancing (phone 203 777-7442 for those who want to
investigate).

Barry Bolding and Jeyapandian Kottalam at Cray and Steve Chin at 
IBM have remarked that the shortness of the benchmarks (100 steps 
each) cause a greater percentage of time to be spent in scalar 
setup of the job as speeds increase. This is quite a valid point: 

       free energy calculation on DNA in vacuum

          --- time (sec) ---
          startup      total         % startup

HP 735	     3	        174          1.7
Cray C90     2           66          3.0

The free energy perturbation uses about twice the runtime
as plain dynamics because of multiple energy evaluations, 
but it has to set up more as well. Clearly the percentage 
of setup time is worth considering further; my immediate 
reaction has been to increase the number of steps to 500. 
Thomas Huber of Ludwig Maximilian Universitaet, Muenchen 
has suggested subtracting the time for a 100 step run from 
a 200 step run, and perhaps this is what I'll do instead. I
don't see any need for thousands of steps unless it proves
necessary to average out fluctuations in the rest of the
computer system - which may be required for the message-
passing architectures, especially workstation clusters (the
results I have reported so far are rather reproducible).
Much more important in the general case is to vary the cutoff
distance.

At this point I'd like to thank again the growing list of
people who have contributed by discussion or remark to my 
formulation and interpretation of the benchmarks. In 
approximate temporal order: George Seibel (now at Smith Kline), 
Dave Case (Scripps), Steve DeBolt (Scripps), Steven Chin (IBM), 
Thomas Huber (Ludwig Maximilian Universitaet, Muenchen), 
Richard Walsh (Minnesota Supercomputer Center, Inc.), Barry 
Bolding (Cray) and Jeyapandian Kottalam (Cray).

Bill Ross