A group of Cambridge (UK) computer scientists have set a new gold standard for openness and reproducibility in research by sharing the more than 200 GB of data and 20,000 lines of code behind their latest results - an unprecedented degree of openness in a peer-reviewed publication. The researchers hope that this new gold standard will be adopted by other fields, increasing the reliability of research results, especially for work which is publicly funded.
The researchers are presenting their results at a talk today at the 12th USENIX Symposium on Networked Systems Design and Implementation (NSDI) in Oakland, California.
In recent years there's been a great deal of discussion about so-called 'open access' publications - the idea that research publications, particularly those funded by public money, should be made publicly available.
Computer science has embraced open access more than many disciplines, with some publishers sub-licensing publications and allowing authors to publish them in open archives. However, as more and more corporations publish their research in academic journals, and as academics find themselves in a 'publish or perish' culture, the reliability of research results has come into question.
http://phys.org/news/2015-05-gold-standard.html
[Also Covered By]: http://www.eurekalert.org/pub_releases/2015-05/uoc-ngs043015.php
[Source]: http://www.cam.ac.uk/research/news/new-gold-standard-established-for-open-and-reproducible-research
(Score: 3, Interesting) by zeigerpuppy on Tuesday May 05 2015, @01:09AM
there are some excellent resources for reproducible research;
particularly this book: https://www.crcpress.com/product/isbn/9781466572843 [crcpress.com]
(Score: 3, Insightful) by bradley13 on Tuesday May 05 2015, @06:18AM
I don't get it either. Back when I was active in research, I published my code and data online, for anyone to download. IIRC my entire research group at UT Austin did the same. Science is all about reproducible results; in computer science, reproducing results requires access to the underlying code and data.
Publishing your code and data has been voluntary up to now. The only thing that needs to change: all serious journals need to require full disclosure and put the links right in the articles. Referees should check that the disclosed information is available and complete.
One problem remains: people move around a lot, and leave a trail of data tied to old accounts that are eventually deleted, or at least no longer maintained. For published articles, a copy of the code and data should be hosted by the conference or by the publishers, so that it remains available long-term. Hey, the journals would actually be providing a tangible service, in return for their crazy prices!
Everyone is somebody else's weirdo.