At the Sackler Colloquium focused on “Reproducibility of Research: Issues and Proposed Remedies,” David B. Allison, Richard Shiffrin, and Victoria Stodden invited discussion on the main topics of defining reproducibility in various research contexts and providing remedies that contribute to greater reproducibility and transparency.
A dozen articles shaping the colloquium were published in a special issue in March 2018 of Proceedings of the National Academy of Sciences and covered a comprehensive range of subjects from the collection of data to the dissemination of findings from both the scientific and non-scientific communities. Article featured in the PNAS special issue are:
- “Empirical Confidence Interval Calibration for Population-Level Effect Estimation Studies in Observational Healthcare Data,” by Martijn J. Schuemie, George Hripcsak, Patrick B. Ryan, David Madigan and Marc A. Suchard.
- “Training Replicable Predictors in Multiple Studies,” by Prasad Patil and Giovanni Parmigiani.
- “An Empirical Analysis of Journal Policy Effectiveness for Computational Reproducibility,” by Victoria Stodden, Jennifer Seiler and Zhaokun Ma.
- “Standards for Design and Measurement Would Make Clinical Research Reproducible and Usable,” by Kay Dickersin and Evan Mayo-Wilson.
- “Enhancing Primary Reports of Randomized Controlled Trials: Three Most Common Challenges and Suggested Solutions,” by Guowei Li et al.
- “The Preregistration Revolution,” by Brian A. Nosek, Charles R. Ebersole, Alexander C. DeHaven, and David T. Mellor.
- “Metastudies for Robust Tests of Theory,” by Beth Baribault et al.
- “Misrepresentation and Distortion of Research in Biomedical Literature,” by Isabelle Boutron and Philippe Ravaud.
- “Crisis or Self-correction: Rethinking Media Narratives about the Well-being of Science,” by Kathleen Hall Jamieson.
- “Is Science Really Facing a Reproducibility Crisis, and Do We Need It To?,” by Daniele Fanelli.
Overall, the authors and participants were interested in identifying challenges and limitations to rigor and reproducibility, and in developing paths forward to make science more rigorous and reproducible: what are incremental and structural changes which can be implemented now to improve the present condition of scientific investigations and publications to fulfil the mantra that science is self-correcting?