From owner-chemistry \\at// ccl.net Mon Oct 17 13:12:01 2011 From: "Sergio Manzetti sergio.manzetti]~[gmail.com" To: CCL Subject: CCL:G: Science code manifesto Message-Id: <-45663-111017120314-12798-lpniXhCdMNwtUyzmWHAH3g_-_server.ccl.net> X-Original-From: Sergio Manzetti Content-Type: multipart/alternative; boundary=20cf307c9b26052b2704af80bebc Date: Mon, 17 Oct 2011 18:03:05 +0200 MIME-Version: 1.0 Sent to CCL by: Sergio Manzetti [sergio.manzetti|-|gmail.com] --20cf307c9b26052b2704af80bebc Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Jim, althoughit may be a kiss of life, I agree with Sebastian. The availibility of tools does not mean that it is available to any scientist as long as a manifesto governs the work and responsibility of a journal to identify the work as correct or not. The reproducibility is enitrely in the hands of the journals, and a manifesto as the one Brian suggests does not replace the journals responsibility or change the skills of a reviewer. As any other manifesto, itattempts to trap principle in time= , but with time any principle dissolves into new hands and new generations. The ultimate responsibility is in the reviewers to request information, and if they do not, the editor should be aware of the quality of the reviewers work. Sergio On Mon, Oct 17, 2011 at 3:15 PM, Jim Kress ccl_nospam~~kressworks.com < owner-chemistry^ccl.net> wrote: > > Sent to CCL by: "Jim Kress" [ccl_nospam .. kressworks.com] > Although it may be the kiss of death, I agree with Brian. > > The fundamental requisite of scientific work is that it must be > reproducible by ANY OTHER SCIENTIST skilled in the art. The lack of acce= ss > to the tools necessary to establish reproducibility should obviate the > publication of any scientific paper until such access and detailed > examination is made available. > > Jim Kress > > > -----Original Message----- > > From: owner-chemistry+ccl_nospam=3D=3Dkressworks.com ~~ ccl.net > > [mailto:owner-chemistry+ccl_nospam=3D=3Dkressworks.com ~~ ccl.net] On B= ehalf > > Of Brian Salter-Duke brian.james.duke.:.gmail.com > > Sent: Sunday, October 16, 2011 5:33 PM > > To: Kress, Jim > > Subject: CCL:G: Science code manifesto > > > > > > Sent to CCL by: Brian Salter-Duke [brian.james.duke(~)gmail.com] The > > review of manuscripts by referees prior to publication is just a small > part of > > what this is all about. The main point is about proper review by the > scientific > > community after publication. There, while not everyone has access to a > > 600MHz NMR or has paid to get Gaussian, some people in the community wi= ll > > have. The guys who paid for Gaussian however, may not have bought the > > code and in some cases, it is impossible to buy the code, as they are > > commercial secrets. This manifesto is saying that commercial secrets ar= e > not > > compatible with good science. > > > > Of course the need to this kind of review is not common, but if there a= re > real > > concerns about the work, it is necessary that the scientific community > can > > look seriously at the code to see exactly what the program is doing. > > > > Brian. > > > > On Sun, Oct 16, 2011 at 8:05 PM, Sebastian Kozuch kozuchs^_^yahoo.com > > wrote: > > > I like the idea of open software, and in principle I support the > > > proposal of having access to the software code for the possibility of > > > full review of a theoretical work. However, I feel that this is more > > > wishful thinking than real scientific life. > > > > > > Let=92s say that I receive a manuscript for review that uses program = X, > > > which I may have access to its code now (since I=92m the reviewer). D= o I > > > have to spend a week trying to understand its algorithms to check if > > > the frequencies are correctly calculated? For me, most (if not all) > > > the programs are in practice black boxes, as I=92m hardly a programme= r. > > > Therefore, open or closed software (usually) doesn=92t make me any > > > difference, except from a philosophical perspective. > > > > > > Now, let me consider an analogy from experimental chemistry. I have t= o > > > review a manuscript where the authors tested some compound with a 600 > > > MHz NMR. My lab is much more humble, and I only have access to a 400 > > > MHz NMR. Id est, I cannot reproduce the results of the paper that I= =92m > > > reviewing. Is here any difference compared to the theoretical case? > > > > > > I had in a couple of cases problems to reproduce the results of > > > papers, mostly because the authors didn=92t provide enough informatio= n. > > > However, for 100% reproducibility I need 100% the same conditions (an= d > > > a lot of resources and time). This makes the selection of the softwar= e > just a > > small issue. > > > > > > I would like to hear the thoughts of other people about this issue, a= s > > > I consider the peer reviewing process a very complex and far from > > > perfect system. > > > > > > > > > > > > > > > Sebastian > > > > > > ________________________________ > > > From: Brian.James.Duke{:}gmail.com > > > To: "Kozuch, Sebastian " > > > Sent: Friday, October 14, 2011 11:18 PM > > > Subject: CCL: Science code manifesto > > > > > > I just encountered the Science Code Manifesto, which essentially > > > states that all computer code used for scientific analysis and > > > modeling should be available for review. It appears to have started > > > with the Climate Code Foundation. I encourage you to visit the web > > > site and consider endorsing the Manifesto. > > > > > > http://sciencecodemanifesto.org/ > > > > > > Note that this is not specifying open source code, so GAMESS(US), > > > GAMESS(UK), DALTON etc., as well as open source codes such as PSI3 am= d > > > MPQC satisfies the points of the manifesto. Of course some other > > > quantum chemistry codes do not. I think we should be putting pressure > > > on the authors of such codes to meet the criteria in this manifesto. > > > > > > Brian. > > > > > > > > > > > > > > -- > > Brian Salter-Duke (aka Brian Duke) > > Brian.James.Duke:+:gmail.com> > > > > -=3D This is automatically added to each message by the mailing script = =3D-> > > --20cf307c9b26052b2704af80bebc Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Jim, althoughit=A0 may be a kiss of life, I agree with Sebastian.

Th= e availibility of tools does not mean that it is available to any scientist= as long as a manifesto governs the work and responsibility of a journal to= identify the work as correct or not. The reproducibility is enitrely in th= e hands of the journals, and a manifesto as the one Brian suggests does not= replace the journals responsibility or change the skills of a reviewer. As= any other manifesto, itattempts to trap principle in time, but with time a= ny principle dissolves into new hands and new generations. The ultimate res= ponsibility is in the reviewers to request information, and if they do not,= the editor should be aware of the quality of the reviewers work.

Sergio

On Mon, Oct 17, 2011 at 3:15 P= M, Jim Kress ccl_nospam~~kressworks.com <owner-c= hemistry^ccl.net> wrote:

Sent to CCL by: "Jim Kress" [ccl_nospam .. kressworks.com]
Although it may be the kiss of death, I agree with Brian.

The fundamental requisite of scientific work is that it must be reproducibl= e by ANY OTHER SCIENTIST skilled in the art. =A0The lack of access to the t= ools necessary to establish reproducibility should obviate the publication = of any scientific paper until such access and detailed examination is made = available.

Jim Kress

> -----Original Message-----
> From: owner-chemistry+ccl_nospam=3D=3Dkressworks.com ~~ ccl.net
> [mailto:owner-chemistr= y+ccl_nospam=3D=3Dk= ressworks.com ~~ ccl.net] On Behalf
> Of Brian Salter-Duke brian.james.duke.:.gmail.com
> Sent: Sunday, October 16, 2011 5:33 PM
> To: Kress, Jim
> Subject: CCL:G: Science code manifesto
>
>
> Sent to CCL by: Brian Salter-Duke [brian.james.duke(~)gmail.com] The
> review of manuscripts by referees prior to publication is just a small= part of
> what this is all about. The main point is about proper review by the s= cientific
> community after publication. There, while not everyone has access to a=
> 600MHz NMR or has paid to get Gaussian, some people in the community w= ill
> have. The guys who paid for Gaussian however, may not have bought the<= br> > code and in some cases, it is impossible to buy the code, as they are<= br> > commercial secrets. This manifesto is saying that commercial secrets a= re not
> compatible with good science.
>
> Of course the need to this kind of review is not common, but if there = are real
> concerns about the work, it is necessary that the scientific community= can
> look seriously at the code to see exactly what the program is doing. >
> Brian.
>
> On Sun, Oct 16, 2011 at 8:05 PM, Sebastian Kozuch kozuchs^_^yahoo.com
> <owner-chemistry:+:ccl= .net> wrote:
> > I like the idea of open software, and in principle I support the<= br> > > proposal of having access to the software code for the possibilit= y of
> > full review of a theoretical work. However, I feel that this is m= ore
> > wishful thinking than real scientific life.
> >
> > Let=92s say that I receive a manuscript for review that uses prog= ram X,
> > which I may have access to its code now (since I=92m the reviewer= ). Do I
> > have to spend a week trying to understand its algorithms to check= if
> > the frequencies are correctly calculated? For me, most (if not al= l)
> > the programs are in practice black boxes, as I=92m hardly a progr= ammer.
> > Therefore, open or closed software (usually) doesn=92t make me an= y
> > difference, except from a philosophical perspective.
> >
> > Now, let me consider an analogy from experimental chemistry. I ha= ve to
> > review a manuscript where the authors tested some compound with a= 600
> > MHz NMR. My lab is much more humble, and I only have access to a = 400
> > MHz NMR. Id est, I cannot reproduce the results of the paper that= I=92m
> > reviewing. Is here any difference compared to the theoretical cas= e?
> >
> > I had in a couple of cases problems to reproduce the results of > > papers, mostly because the authors didn=92t provide enough inform= ation.
> > However, for 100% reproducibility I need 100% the same conditions= (and
> > a lot of resources and time). This makes the selection of the sof= tware just a
> small issue.
> >
> > I would like to hear the thoughts of other people about this issu= e, as
> > I consider the peer reviewing process a very complex and far from=
> > perfect system.
> >
> >
> >
> >
> > Sebastian
> >
> > ________________________________
> > From: Brian.James.Duke{:}gmail.com <owner-chemistry()ccl.net>
> > To: "Kozuch, Sebastian " <kozuchs()yahoo.com>
> > Sent: Friday, October 14, 2011 11:18 PM
> > Subject: CCL: Science code manifesto
> >
> > I just encountered the Science Code Manifesto, which essentially<= br> > > states that all computer code used for scientific analysis and > > modeling should be available for review. It appears to have start= ed
> > with the Climate Code Foundation. I encourage you to visit the we= b
> > site and consider endorsing the Manifesto.
> >
> > ht= tp://sciencecodemanifesto.org/
> >
> > Note that this is not specifying open source code, so GAMESS(US),=
> > GAMESS(UK), DALTON etc., as well as open source codes such as PSI= 3 amd
> > MPQC satisfies the points of the manifesto. Of course some other<= br> > > quantum chemistry codes do not. I think we should be putting pres= sure
> > on the authors of such codes to meet the criteria in this manifes= to.
> >
> > Brian.
> >
> >
>
>
>
> --
> Brian Salter-Duke (aka Brian Duke)
> Brian.James.Duke:+:gmai= l.com>



-=3D This is automatically added to each message by the mailing script =3D-=
E-mail to subscribers: CHEMISTRY^ccl.n= et or use:
=A0 =A0 =A0http://www.ccl.net/cgi-bin/ccl/send_ccl_message

E-mail to administrators: CHEM= ISTRY-REQUEST^ccl.net or use
=A0 =A0 =A0http://www.ccl.net/cgi-bin/ccl/send_ccl_message

Subscribe/Unsubscribe:
=A0 =A0 =A0http://www.ccl.net/chemistry/sub_unsub.shtml

Before posting, check wait time at: http://www.ccl.net

Job: http://www.ccl.n= et/jobs
Conferences: http://server.ccl.net/chemistry/announcements/co= nferences/

Search Messages: http://www.ccl.net/chemistry/searchccl/index.shtml
=A0 =A0 =A0h= ttp://www.ccl.net/spammers.txt

RTFI: http://www.ccl.net/chemistry/aboutccl/instructions/





--20cf307c9b26052b2704af80bebc--