It is clear that the issue of the Science Code
Manifesto
stirred a lot of feelings. I must say that I found this topic almost as
enjoyable as the match between Fortran and C++. I learnt a couple of small
things with this discussion, but all in all I still subscribe to the same ideas
I wrote some days ago:
âI like the idea of open software, and in principle
I
support the proposal of having access to the software codeâ However, I feel
that this is more wishful thinking than real scientific life.â
âOpen or closed software (usually) doesnât make
me any
difference, except from a philosophical perspective.â
I think of myself as a pragmatic guy. That means that as
much as I like GPL and open software, I am prepared to pay for a close program
that
works better. And I dare to say most of the computational chemists try not to
dive into the code anyway, as we only want to obtain an energy value, and not
understand how the Roothaanâs equations were implemented.
Let me rescue one thought that appeared (sorry I
donât
remember the name of the author): âReproducibility lies in the possibility
of
obtaining similar values, not necessarily with the same programâ. So, if
for
example I wrote down in the paper the functional and the basis set, that should
be enough for reproducibility of the energy and geometry (although a good
researcher should help his community by giving much more information). If
programs X and Y give very different bond distances, then clearly one of them
is doing wrong its work. Who cares about the code if itâs a crappy code
anyway?
It is not reproducible, and that is death according to the scientific method.
I still think that we must compare ourselves with the
experimental chemistry world, and donât make a special epistemological
rules for
computational science. If I use a specific NMR, nobody will ask me to publish
the designs of the equipment including the Fourier transform algorithm and the
crystal structure of the superconductor (which I donât have anyway); I must
only write down the model and frequency of the machine. Should we ask more from
the theoreticians?
Let me be repetitive:
âI like the idea of open software, and in principle
I
support the proposal of having access to the software codeâ However, I feel
that this is more wishful thinking than real scientific life.â
Now back to my calculations. From time to time we must
stop
arguing and make a bit of real work.
Best,
Sebastian
From: Sebastian Kozuch
kozuchs^_^yahoo.com <owner-chemistry=-Ì.net>
To: "Kozuch, Sebastian "
<kozuchs=-=yahoo.com>
Sent: Sunday, October 16, 2011 11:05 AM
Subject: CCL: Science code
manifesto
I like the idea of open software, and
in principle I support
the proposal of having access to the software code for the possibility of full
review of a theoretical work. However, I feel that this is more wishful
thinking than real scientific life.
Letâs say that I receive a manuscript
for review that uses
program X, which I may have access to its code now (since Iâm the
reviewer). Do
I have to spend a week trying to understand its algorithms to check if the
frequencies are correctly calculated? For me, most (if not all) the programs are
in practice black boxes, as Iâm hardly a programmer. Therefore, open or
closed
software (usually) doesnât make me any difference, except from a
philosophical perspective.
Now, let me consider an analogy from
experimental chemistry.
I have to review a manuscript where the authors tested some compound with a 600
MHz NMR. My lab is much more humble, and I only have access to a 400 MHz NMR.
Id est, I cannot reproduce the results of the paper that Iâm reviewing. Is
here
any difference compared to the theoretical case?
I had in a couple of cases problems to
reproduce the results
of papers, mostly because the authors didnât provide enough information.
However, for 100% reproducibility I need 100% the same conditions (and a lot of
resources and time). This makes the selection of the software just a small
issue.
I would like to hear the thoughts of other
people about this
issue, as I consider the peer reviewing process a very complex and far from
perfect system.
Sebastian
From: Brian.James.Duke{:}gmail.com
<owner-chemistry()ccl.net>
To: "Kozuch, Sebastian "
<kozuchs()yahoo.com>
Sent: Friday, October 14, 2011 11:18
PM
Subject: CCL: Science code
manifesto
I just encountered the Science Code Manifesto, which
essentially states
that all computer code used for scientific analysis and
modeling should
be available for review. It appears to have started with the
Climate
Code Foundation. I encourage you to visit the web site and
consider
endorsing the Manifesto.
http://sciencecodemanifesto.org/Note that this is
not specifying open source code, so GAMESS(US),
GAMESS(UK), DALTON etc., as
well as open source codes such as PSI3 amd
MPQC satisfies the points of the
manifesto. Of course some other quantum
chemistry codes do not. I think we
should be putting pressure on the
authors of such codes to meet the criteria
in this
manifesto.
Brian.