Subject: Re: CCL:Summary: AMD-K6/linux for G98

From: "Gregory A. Landrum" <[email-masked]>
Date: Thu, 25 Feb 1999 15:53:03 +0100 (NFT)

Normally I stay silent on these things, but this comment was just a bit
too indicative of general "benchmark abuse".

> For pinene using DFT/B3LYP with large basis sets, for
>example, the machine i bought for $11,000 was faster than a Cray T3E-600
>with 4 processors and only 15% slower than a Cray T3E-1200 with 4
>processors, both running parallel Gaussian 94. (The G94 calculations were
>done by Sosa and Frisch, so were probably pretty well optimized for the
>machine, see last July 15 number of J. Comput. Chem. Sosa et al. Vol 19,
>p. 1053). I haven't priced those machines but they must be >$100k. If you
>want to benchmark against one of your own machines, Pulay would be happy
>to do some calculations for you.

The comparison here is *completely* invalid.  If you want to compare
the speed of machines, you must be running the same program.  In this
case, a highly-optimized DFT implementation (the one from Pulay) is being
compared to an implementation which can charitably be called "not so
great", this does not say a single thing about the relative speeds of the
computers running those programs.  

Having Pulay do a benchmark for you would be altering two variables at the
same time:
 (a) the hardware
 (b) the software
it is impossible to decouple the effects of the two changes.
One more time, for good measure:
If you want reliable benchmarks of *hardware* performance, you must
compare the same programs on the different hardware.

-greg

-- 
---------------------
Dr. Greg Landrum  ([email-masked])
Institute of Inorganic Chemistry
Aachen University of Technology
Prof.-Pirlet-Str. 1, D-52074 Aachen, Germany
Phone: 049-241-80-7004
Fax: 049-241-8888-288