CCL:G: Science code manifesto

[Laurence Cuffe <cuffe]~[mac.com>]

 On Oct 17, 2011, at 08:15 AM, "JoÃo BrandÃo
 jbrandao+/-ualg.pt" <owner-chemistry+/-ccl.net> wrote:
>  Sorry, but I disagree.
>  "but if there are real concerns about the work, it is necessary that the
>  scientific
>  community can look seriously at the code to see exactly what the program is
>  doing."
>  In my opinion:
>  If anyone has real concerns about any work, the best way for science development
>  is to write (or use) a different code and compare the results.
>  JoÃo BrandÃo
 It has happened previously that the same calculation run on two different
 software packages has resulted in different results. This can happen when
 different teams interpret research methods differently.  If we can examine the
 code, then we can see what has been done.  If we cannot examine the code, we
 have no basis other than reputation on which to decide which is correct.
 Basing decisions on reputation only,ï makes for poor science.
 All the best
 Laurence Cuffe.
>  Em 16-10-2011 22:33, Brian Salter-Duke brian.james.duke.:.gmail.com escreveu:
> >  Sent to CCL by: Brian Salter-Duke [brian.james.duke(~)gmail.com]
> >  The review of manuscripts by referees prior to publication is just a
> >  small part of what this is all about. The main point is about proper
> >  review by the scientific community after publication. There, while not
> >  everyone has access to a 600MHz NMR or has paid to get Gaussian, some
> >  people in the community will have. The guys who paid for Gaussian
> >  however, may not have bought the code and in some cases, it is
> >  impossible to buy the code, as they are commercial secrets. This
> >  manifesto is saying that commercial secrets are not compatible with
> >  good science.
> >  Of course the need to this kind of review is not common, but if there
> >  are real concerns about the work, it is necessary that the scientific
> >  community can look seriously at the code to see exactly what the
> >  program is doing.
> >  Brian.
> >  On Sun, Oct 16, 2011 at 8:05 PM, Sebastian Kozuch kozuchs^_^yahoo.com
> >  <owner-chemistry:+:ccl.net> wrote:
> > >  I like the idea of open software, and in principle I support the proposal of
> > >  having access to the software code for the possibility of full review of a
> > >  theoretical work. However, I feel that this is more wishful thinking than
> > >  real scientific life.
> > >  Letâs say that I receive a manuscript for review that uses program X, which
> > >  I may have access to its code now (since Iâm the reviewer). Do I have to
> > >  spend a week trying to understand its algorithms to check if the frequencies
> > >  are correctly calculated? For me, most (if not all) the programs are in
> > >  practice black boxes, as Iâm hardly a programmer. Therefore, open or closed
> > >  software (usually) doesnât make me any difference, except from a
> > >  philosophical perspective.
> > >  Now, let me consider an analogy from experimental chemistry. I have to
> > >  review a manuscript where the authors tested some compound with a 600 MHz
> > >  NMR. My lab is much more humble, and I only have access to a 400 MHz NMR. Id
> > >  est, I cannot reproduce the results of the paper that Iâm reviewing. Is
> > >  here
> > >  any difference compared to the theoretical case?
> > >  I had in a couple of cases problems to reproduce the results of papers,
> > >  mostly because the authors didnât provide enough information. However, for
> > >  100% reproducibility I need 100% the same conditions (and a lot of resources
> > >  and time). This makes the selection of the software just a small
> > >  issue