Reproducible Research

• In 1994, Jon Claerbout was one of the first scientists to emphasize that computational methods threaten the reproducibility of research unless open access is provided to both the data and the software underlying a publication.
• Scientific Errors and Misconduct “An Array of Errors” - The Economist 2011
• Open Science movement Center for Open Science and Open Science Framework
• Becoming the norm - the Biostatistics journal now marks every article as to the extent they are “reproducible”

Reproducible Research

Our reproducible research policy is for papers in the journal to be kite-marked D if the data on which they are based are freely available, C if the authors’ code is freely available, and R if both data and code are available, and our Associate Editor for Reproducibility is able to use these to reproduce the results in the paper. Data and code are published electronically on the journal’s website as Supplementary Materials.

• Victoria Stodden’s definitions of types of reproducibility

Dynamic Documentation

• “Literate Programming” coined by Donald Knuth in 1984.

“Instead of imagining that our main task is to instruct a computer what to do, let us concentrate rather on explaining to human beings what we want a computer to do.”

• A document that is “dynamic” is created using current data + code + text
• This presentation is an R Markdown presentation. Markdown is a simple formatting syntax for authoring HTML, PDF, and MS Word documents. For more details on using R Markdown see http://rmarkdown.rstudio.com.
• This presentation is “dynamic” and was created by clicking the Knit button.
• The terms most often used are sweave and knit Sweave by Friedrich Leisch and knitr by Yihui Xie
• We will focus on knitr in these R Workshops.
• NO MORE CUT AND PASTE EVER Video

Version Control and Collaboration

• Git
• Github
• Track all versions - changes and updates
• Easy to “roll back”
• Easy to collaborate with multiple team members
• Crowdsourcing & SPEED “Open-Source Genomic Analysis of Shiga-Toxin–Producing E. coli O104:H4”

“We released these data into the public domain under a Creative Commons 0 license, which elicited a burst of crowd-sourced, curiosity-driven analyses carried out by bioinformaticians on four continents. Twenty-four hours after the release of the genome, it had been assembled; 2 days after its dissemination, it had been assigned to an existing sequence type. Five days after the release of the sequence data, we had designed and released strain-specific diagnostic primer sequences, and within a week, two dozen reports had been filed on an open-source wiki (a Web site that facilitates collaborative effort) dedicated to analysis of the strain.”

• the Github Wiki listed above

Example R Code with Output

summary(cars)

##      speed           dist       
##  Min.   : 4.0   Min.   :  2.00  
##  1st Qu.:12.0   1st Qu.: 26.00  
##  Median :15.0   Median : 36.00  
##  Mean   :15.4   Mean   : 42.98  
##  3rd Qu.:19.0   3rd Qu.: 56.00  
##  Max.   :25.0   Max.   :120.00

R code with Output Plot

plot(pressure)