Subject: [Fwd: Re: Athlon SDR/DDR stats for *specific* gaussian98 jobs]

From: [email-masked]
Date: Thu, 03 May 2001 22:16:53 +0200

-------- Original Message --------
From: Velocet <[email-masked]>
Subject: Re: Athlon SDR/DDR stats for *specific* gaussian98 jobs
To: "Robert G. Brown" <[email-masked]>
CC: [email-masked]

On Wed, May 02, 2001 at 06:16:51PM -0400, Robert G. Brown's all...

> IIRC, somebody on the list (Josip Loncaric?) inserted prefetching into
> at least parts of ATLAS for use with athlons back when they were first
> released.  It apparently made a quite significant difference in
> performance.

Oh, btw, I did include Athlon3DNow!2 stuff in my ATLAS just for a laugh.
I then compared my results to the original non ATLAS g98 and everything
matched up for all 4 jobs I chose. I dont think its actually used
for G98 since its single precision stuff only IIRC. 

Also, note, we may start using MPQC in a bit - Graydon Hoare (ex Berlin
project, now at Redhat) is hacking on it to speed it up a bunch if
possible. I havent had a chance to benchmark that (it may demand
TB1333's on DDR boards and Ill be sunk! :)

In fact my stats indicated that a Duron 750 was slightly better for the money
in most cases, but the 900 and 700 are so close, I went with the faster chip
in the hopes that other situations that I cant benchmark now but will
encounter later will favour the faster CPU (also the supply of D700s is
waning).

> in the marginal area where it makes sense to go single.  It may not be
> very easy to tell which >>one<< is truly cost optimal, though, without
> benchmarking your particular code and doing a careful cost comparison
> including hidden costs (e.g. the fact that electricity costs and space
> costs may be 60% higher for lots of singles, each single requires a
> case, its own memory and copy of the OS and network card, and so forth).
> In many cases a dual is only 0.7-0.8 the cost of two singles, although
> the high cost of DDR makes that unlikely in this case.  Still, it is
> under $1/MB, which isn't all THAT bad -- PC133 cost that much only
> months ago.  Months from now DDR may cost little more than SDRAM in
> equivalent amounts.

I started forming my anti SMP bias when I ran a bunch of old G94 jobs
on a dual Celeron board. It *SUCKED* :) 100 Mhz ram was probably
the bottleneck, and the resource locking in Linux 2.2.(early) was not
as nice as it is now in 2.4 (so I hear). I have stats for it actually:

two jobs
""""""""
(Mhz ratio = 1.00 for 550Mhz, times for 2 jobs to be run)

#
C                                                 
P                                         speed    MHZ  efficiency
U CPU           MHz/bus RAM       total   ratio   ratio (mhz/speed ratios)
------------------------------------------------------------------------------
2 C366A         550/100 128M      4293.6  1.00    2.00	   0.50
1 P2 400        400/100 128M      6105.3  0.70    0.73	   0.96
1 P3 450        450/100 256M      5675.0  0.76    0.83	   0.92
1 C300A         450/100 64M       7253.5  0.59    0.83	   0.71

For g94 I saw almost the exact same performance out of a C450 as
a P3-450 (I guess the CPU's SSE extensions were not used by it).
So seeing that my heavily overclocked 550MHz Celeron's were only some
25% faster overall for 2 CPUs vs 1 at 450, thats pretty bad. :)
I think that this is the software fighting with the locking and the
shared bus (Abit BP6 IIRC was the board, which probably didnt have
the fastest architecture either. IIRC, dual celeron was a bad hack).
(As well, the stats above are not that fair as I ran a single job
instead of the 4 different types Im using now as my base set, not to
mention its G94 (outdated) and non-ATLAS.)

So its probably not optimal to even consider stats for SMP from a dual
Celerons considering they're so not designed for it. :) Nonetheless,
my benchmarking of various jobs showed me that memory bandwidth is really
important for most gaussian jobs, and you'd really need a speedy memory
bus to keep up with this. DDR will probably really help SMP for this
kind of thing.

> Sounds like you are dead right on all of this.  Embarrassingly parallel
> jobs, few to no communications, purely CPU bound -- Durons (or whatever
> currently delivers the most raw flops for the least money) are likely to
> be perfect for you.  And for many others, actually.  For a long time I
> like Celerons (or even dual Celerons) for the same reasons, although at
> this point I've converted to AMD-based systems as their cost-benefit has
> overwhelmed Intel's whole product line for my code.

Its interesting however, to note that the improvements per cycle especially
for DDR boards.  I actually removed this stat from my charts, but I did
compare all the Athlons to a Celeron 450 G89-atlas setup. 

These values are MUCH better than the non atlas values - ie atlas
really improves the efficiency of the jobs, especially for newer CPUs -
I assume by making better use of large L1/L2 caches and filling the longer
pipelines more optimally.

efficiency/cycle, atlas g98:
(each column is normalized to 1.00 for C450 w.r.t that column's data)

		   job 1	   job 2	   job 3	   job 4
			      |		      |		      |
		non	atlas |	non	atlas |	non	atlas |	non	atlas
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|~~~~~~~~~~~~~~~|~~~~~~~~~~~~~~~|~~~~~~~~~~~~~~
C450		1.00	1.00  |	1.00	1.00  |	1.00	1.00  |	1.00	1.00
A700			1.05  |	0.88	1.05  |	1.26	1.31  |	1.34	1.39
D750		0.86	1.30  |	0.70	0.77  |	1.05	1.10  |	1.25	1.24
T900		0.89	1.31  |	0.81	0.86  |	1.05	1.08  |	1.23	1.20
T1200DDR		1.08  |		1.04  |		1.30  |		1.42

I think we see these patterns because without optimization for caches
and pipeline, the disparate speeds between the CPU and RAM for the non
DDR machines is very large (up to 4 or 5 CPU cycles/ram cycle). The cache
helps a fair bit (the Tbird and Athlon fare much better than the duron),
but when we get ATLAS involved, the improvements are quite noticeable
over the baseline C450.
 
> Not yet, but maybe soon.  The fast Tbirds do require a big "certified"
> power supply, but I'm guessing they draw a lot less than they "require"
> except maybe in bursts.  I'm betting they draw around 100-150W running,
> a number that recently got some support on the list.

Bursts of CPU usage? Arent all our clusters all hammering our CPUs as much as
possible? And if ATLAS is really doing its job, arent we hammering all parts
of the CPU as much as possible? :)

> Not a preset package, but I'm trying to start a collection of sorts:
> 
>   http://www.phy.duke.edu/brahma/dual_athlon/tests.html

Will check it out.

> Hope this all helps or is interesting.  I'm very interested in Athlon
> performance profiles as they seem to be the current
> most-CPU-for-the-least-money winners, and when one buys in bulk (as
> beowulf humans tend to do) this sort of optimization really matters.

What kind of deals can you get in bulk? From AMD themselves? do you
need to be a big university and have a big press release event to
get these deals from them? How many do you need to get a batch deal?
How deep is the discount?

/kc

> 
>    rgb
> 
> -- 
> Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
> Duke University Dept. of Physics, Box 90305
> Durham, N.C. 27708-0305
> Phone: 1-919-660-2567  Fax: 919-660-2525     email:[email-masked]
> 
> 
> 

-- 
Ken Chase, [email-masked]  *  Velocet Communications Inc.  *  Toronto, CANADA 

_______________________________________________
Beowulf mailing list, [email-masked]
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf

---

From: "M. Nicklaus" <[email-masked]>
Date: Thu, 3 May 2001 18:29:09 -0400 (Eastern Daylight Time)

For what it's worth:  Here's a recent small series of benchmarks we've run
on various Linux systems, plus a Cray and SGI Origin added for comparison.

Program:        Gaussian 98 Rev. A.7.  All executables exactly identical
                for the Linux systems (copied between machines), compiled
                w/ PGI v. 3.2 (on Linux), G.98 standard compilation (no
                tuning/hacking).

Jobs run:       G.98 test jobs # 1, 28, 94, 155, 194, 296, 302, aggregate
                time, as reported in output, all runs single-CPU.

CPU   Speed  Chipset  Kernel   Distro   glibc    Memory    HD/Contr.  time
                                               min. 256MB  MHz, RPM  (sec)
                                                         (all: U-ATA/)

P 4  1.5GHz Int. 850  2.4.2-2  RH 7.1   2.2.2-10 RDRAM-800 100, 7200   278
Athlon 1.33 VIA 686B  2.4.0   SuSE 7.1  2.2-7    DDR 266    66, 7200   299
P III 1 GHz Int. 815  2.4.3   (RH 6.1)  2.1.2    SDRAM 133 100, 7200   497
P III 1 GHz Int. 815  2.4.3   (RH 6.1)  2.1.2    SDRAM 133 100, 5400   497
P III 1 GHz Int. 815  2.2.12   RH 6.1   2.1.2    SDRAM 133 100, 7200   646
P III 1 GHz Int. 815  2.2.12   RH 6.1   2.1.2    SDRAM 133 100, 5400   646
P III 866MHz S/Works  2.4.3   (RH 6.2)  2.1.3-15 SDRAM 133  33, 7200   453
P III 866MHz S/Works  2.2.14   RH 6.2   2.1.3-15 SDRAM 133  33, 7200   526
SGI Origin 2000                                                        921
Cray SV-1                                                             2400

"(...)" in the Distro column denotes newer kernel than out-of-the-box.

Going from a 2.2 kernel to a 2.4 kernel seems to speed things up
considerably, at least on these systems.  Hard drive and IDE
controller speeds appear to be near irrelevant for these jobs.

Marc

------------------------------------------------------------------------
 Marc C. Nicklaus, Ph.D.                 NIH/NCI at Frederick
 E-mail: [email-masked]               Bldg 376, Rm 207
 Phone:  (301) 846-5903                  376 Boyles Street
 Fax:    (301) 846-6033                  FREDERICK, MD 21702      USA
          Head, Computer-Aided Drug Design MiniCore Facility
     Laboratory of Medicinal Chemistry, Center for Cancer Research,
 National Cancer Institute at Frederick, National Institutes of Health
       http://rex.nci.nih.gov/RESEARCH/basic/medchem/mcnbio.htm
------------------------------------------------------------------------