Subject: Re: CCL:databases, archives, and perhaps XML repositories

From: "David E. Bernholdt" <[email-masked]>
Date: Tue, 25 May 1999 12:18:20 -0400

On Mon, 24 May 1999 19:03:27 -0500 (CDT)  Gary Kedziora wrote:
> I know that the Ecce program is an interesting and useful interface 
> to a database of quantum chemistry information.

Minor correction: Ecce (extensible computational chemistry
environment) is really a "problem solving environment" for chemistry
which happens to be designed to store results in a database system.
IMHO databasing here is useful, but secondary.  For more information
about Ecce see http://www.emsl.pnl.gov:2080/docs/ecce/

Dave Feller at PNNL has also been working on a database of QC
results.  I seem to recall him announcing an online prototype here
some time back.  Carl Mellius (sp?) at Sandia-Livermore used to have
an online database of his calculations.

> Here is a proposal.  We could create our own repositories of 
> program output in XML (or some other suitable format) under
> the control of our HTML server.  Then robot programs could be
> developed to go out and look for that information, retrieve it,
> and put it into a form suitable for a database.

I suspect that this will happen naturally to some extent if/when
programs start producing suitably marked-up output.  

My biggest concern about any such database is validation of the
contents.  Simply crawling the web for new results is practically an
invitation for people to maliciously or otherwise inject erroneous
data.  What if someone has tweaked the code and does not document it
properly in the file that's put out for public consumption?

>From my discussions with Dave Feller, he works very hard at validating
information before archiving it (even including basis sets for the
EMSL Basis Set Library).  Otherwise, I think you need to know a fair
amount about the provenance of a calculation before you can decide how
trustworthy it is, and different people may have different ideas about
that. There seems to be a big difference between producing a database
and producing a database you can trust.

Now one interesting scheme would be some kind of automatic validation.
To be useful, this published information is going to have to
completely describe the calculation if it is going to be useful.  So
in principle, the web crawler that picks it up as a candidate for the
database could first put it into a queue for validation.  This would
require repeating the calculation independently to verify the result.
The "distributed.net" model, which has been used to solve a number of
cryptographic problems in a robust, distributed, "background" fashion
is very interesting, though there are obvious problems trying to port
this model to the chemistry community.

David
--
David E. Bernholdt                      | Email:  [email-masked]
Northeast Parallel Architectures Center | Phone:  +1 315 443 3857
111 College Place, Syracuse University  | Fax:    +1 315 443 1973
Syracuse, NY 13244-4100                 | URL:    http://www.npac.syr.edu