From owner-chemistry@ccl.net Mon Oct 17 11:38:00 2011 From: "Herbert Fruchtl herbert.fruchtl{}st-andrews.ac.uk" To: CCL Subject: CCL:G: Science code manifesto Message-Id: <-45661-111017113509-11551-nG3Pwo15+Uxzf8UEA5/YBA::server.ccl.net> X-Original-From: Herbert Fruchtl Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8; format=flowed Date: Mon, 17 Oct 2011 16:36:02 +0100 MIME-Version: 1.0 Sent to CCL by: Herbert Fruchtl [herbert.fruchtl a st-andrews.ac.uk] Strictly speaking, it's reproducible if the method is competely published. You won't sit down and code it to check the results in one paper, but if a method is useful, sooner or later somebody else will write another program implementing it. Luckily, software patents haven't taken root in computational chemistry yet. I seem to remember that in the early days of DFT, commercial programs with closed-source functionals started appearing. I don't know if they are still in use in some industrial settings, but results obtained with them would certainly not be publishable. Herbert On 17/10/11 14:15, Jim Kress ccl_nospam~~kressworks.com wrote: > > Sent to CCL by: "Jim Kress" [ccl_nospam .. kressworks.com] > Although it may be the kiss of death, I agree with Brian. > > The fundamental requisite of scientific work is that it must be reproducible by ANY OTHER SCIENTIST skilled in the art. The lack of access to the tools necessary to establish reproducibility should obviate the publication of any scientific paper until such access and detailed examination is made available. > > Jim Kress > >> -----Original Message----- >> From: owner-chemistry+ccl_nospam==kressworks.com ~~ ccl.net >> [mailto:owner-chemistry+ccl_nospam==kressworks.com ~~ ccl.net] On Behalf >> Of Brian Salter-Duke brian.james.duke.:.gmail.com >> Sent: Sunday, October 16, 2011 5:33 PM >> To: Kress, Jim >> Subject: CCL:G: Science code manifesto >> >> >> Sent to CCL by: Brian Salter-Duke [brian.james.duke(~)gmail.com] The >> review of manuscripts by referees prior to publication is just a small part of >> what this is all about. The main point is about proper review by the scientific >> community after publication. There, while not everyone has access to a >> 600MHz NMR or has paid to get Gaussian, some people in the community will >> have. The guys who paid for Gaussian however, may not have bought the >> code and in some cases, it is impossible to buy the code, as they are >> commercial secrets. This manifesto is saying that commercial secrets are not >> compatible with good science. >> >> Of course the need to this kind of review is not common, but if there are real >> concerns about the work, it is necessary that the scientific community can >> look seriously at the code to see exactly what the program is doing. >> >> Brian. >> >> On Sun, Oct 16, 2011 at 8:05 PM, Sebastian Kozuch kozuchs^_^yahoo.com >> wrote: >>> I like the idea of open software, and in principle I support the >>> proposal of having access to the software code for the possibility of >>> full review of a theoretical work. However, I feel that this is more >>> wishful thinking than real scientific life. >>> >>> Let’s say that I receive a manuscript for review that uses program X, >>> which I may have access to its code now (since I’m the reviewer). Do I >>> have to spend a week trying to understand its algorithms to check if >>> the frequencies are correctly calculated? For me, most (if not all) >>> the programs are in practice black boxes, as I’m hardly a programmer. >>> Therefore, open or closed software (usually) doesn’t make me any >>> difference, except from a philosophical perspective. >>> >>> Now, let me consider an analogy from experimental chemistry. I have to >>> review a manuscript where the authors tested some compound with a 600 >>> MHz NMR. My lab is much more humble, and I only have access to a 400 >>> MHz NMR. Id est, I cannot reproduce the results of the paper that I’m >>> reviewing. Is here any difference compared to the theoretical case? >>> >>> I had in a couple of cases problems to reproduce the results of >>> papers, mostly because the authors didn’t provide enough information. >>> However, for 100% reproducibility I need 100% the same conditions (and >>> a lot of resources and time). This makes the selection of the software just a >> small issue. >>> >>> I would like to hear the thoughts of other people about this issue, as >>> I consider the peer reviewing process a very complex and far from >>> perfect system. >>> >>> >>> >>> >>> Sebastian >>> >>> ________________________________ >>> From: Brian.James.Duke{:}gmail.com >>> To: "Kozuch, Sebastian " >>> Sent: Friday, October 14, 2011 11:18 PM >>> Subject: CCL: Science code manifesto >>> >>> I just encountered the Science Code Manifesto, which essentially >>> states that all computer code used for scientific analysis and >>> modeling should be available for review. It appears to have started >>> with the Climate Code Foundation. I encourage you to visit the web >>> site and consider endorsing the Manifesto. >>> >>> http://sciencecodemanifesto.org/ >>> >>> Note that this is not specifying open source code, so GAMESS(US), >>> GAMESS(UK), DALTON etc., as well as open source codes such as PSI3 amd >>> MPQC satisfies the points of the manifesto. Of course some other >>> quantum chemistry codes do not. I think we should be putting pressure >>> on the authors of such codes to meet the criteria in this manifesto. >>> >>> Brian. >>> >>> >> >> >> >> -- >> Brian Salter-Duke (aka Brian Duke) >> Brian.James.Duke:+:gmail.com> > -- Herbert Fruchtl Senior Scientific Computing Officer School of Chemistry, School of Mathematics and Statistics University of St Andrews -- The University of St Andrews is a charity registered in Scotland: No SC013532