I like the idea of open software, and in principle
I support
the proposal of having access to the software code for the possibility of full
review of a theoretical work. However, I feel that this is more wishful
thinking than real scientific life.
Letâs say that I receive a manuscript for review
that uses
program X, which I may have access to its code now (since Iâm the
reviewer). Do
I have to spend a week trying to understand its algorithms to check if the
frequencies are correctly calculated? For me, most (if not all) the programs are
in practice black boxes, as Iâm hardly a programmer. Therefore, open or
closed
software (usually) doesnât make me any difference, except from a
philosophical perspective.
Now, let me consider an analogy from experimental
chemistry.
I have to review a manuscript where the authors tested some compound with a 600
MHz NMR. My lab is much more humble, and I only have access to a 400 MHz NMR.
Id est, I cannot reproduce the results of the paper that Iâm reviewing. Is
here
any difference compared to the theoretical case?
I had in a couple of cases problems to reproduce the
results
of papers, mostly because the authors didnât provide enough information.
However, for 100% reproducibility I need 100% the same conditions (and a lot of
resources and time). This makes the selection of the software just a small
issue.
I would like to hear the thoughts of other people about
this
issue, as I consider the peer reviewing process a very complex and far from
perfect system.
Sebastian
From:
Brian.James.Duke{:}gmail.com <owner-chemistry(a)ccl.net>
To: "Kozuch, Sebastian "
<kozuchs(a)yahoo.com>
Sent: Friday, October 14, 2011 11:18 PM
Subject: CCL: Science code
manifesto
I just encountered the Science Code Manifesto, which
essentially states
that all computer code used for scientific analysis and
modeling should
be available for review. It appears to have started with the
Climate
Code Foundation. I encourage you to visit the web site and
consider
endorsing the Manifesto.
http://sciencecodemanifesto.org/Note that this is
not specifying open source code, so GAMESS(US),
GAMESS(UK), DALTON etc., as
well as open source codes such as PSI3 amd
MPQC satisfies the points of the
manifesto. Of course some other quantum
chemistry codes do not. I think we
should be putting pressure on the
authors of such codes to meet the criteria
in this manifesto.
Brian.