The article ‘Never waste a good crisis: Confronting Reproducibility in Translational Research’ by Daniel Drucker and published in Cell Metabolism picks up many well-known and extensively discussed issues concerning the reproducibility crisis, but also introduced some really interesting ideas which could be game changers for the scientific community. Drucker is a scientist working in the field of cell metabolism and he gives impressive examples of different factors which then lead to unreproducible data, like cell line authentication, antibody characterization, several aspects of how to include proper controls in different experimental settings and also the lack of showing negative data. However, , two additional and so far not widely discussed ideas of Drucker shall be mentioned here: First, he proposes that meetings should have dedicated sessions to discuss the issue of unreproducible data which could create awareness of particular unreproducible experiments in each research field. Second and with much broader implications, he proposes to establish a novel index, the Reproducibility index (R-Index) and suggests to count the papers of a scientist that were reproduced by other groups.
Drucker acknowledges that this is indeed a challenge, starting with a proper definition of what reproducibility of a paper exactly means. However, once established, it could help breaking new ground by finding innovative measures to judge the research output of scientists. In any case, the R-Index would give a more direct rating of the quality of research which is completely missing when using current evaluation methods based solely on the Journal’s Impact Factor.
Reproducibility: Respect your cells! Working with cell cultures seems to be pretty straight forward – at least most times – and is done routinely in many in vitro labs. However, the recent article by Monya Baker points out many pitfalls when working with cell cultures and shows that the devil can be in the detail. Most scientists will agree that the “growth serum” supplemented to the cell culture medium is a huge source for variations and proper characterisation of each batch is needed to be able to judge its effect on the cell line of interest. But what about cell line authentication? The effect of light on the cell culture medium? The influence of the level of medium above the cell culture during various stages of the experiment? There are so many different aspects that can influence the outcome of an experiment and just being aware of them is already an important step forward to increase the reproducibility of cell culture experiments.
A Framework for Improving the Quality of Research in the Biological Sciences. In this editorial, Casadevall et al. summarized the six recommendations how to improve the quality of research in the biological sciences made by participants of The American Academy of Microbiology colloquium held end of last year: 1. Design rigorous and comprehensive evaluation criteria to recognize and reward high-quality scientific research; 2. Require universal training in good scientific practices, appropriate statistical usage, and responsible research practices for scientists at all levels; 3. Establish open data at the timing of publication as the standard operating procedure throughout the scientific enterprise; 4. Encourage scientific journals to publish negative data that meet methodologic standards of quality; 5. Agree upon common criteria among scientific journals for retraction of published papers, to provide consistency and transparency; 6. Strengthen research integrity oversight and training.
Especially the first point (Evaluation criteria to recognize high-quality research) is very interesting as it introduces the development of a metric to evaluate scientific quality (based on appropriateness of statistical analysis, quality of the methods, validation of key research tools and reagents, etc.), which provides a quality indicator in real time and does not depend on Journal Impact Factors to judge the productivity or influence of individual scientists.
The Preclinical Reproducibility & Robustness (PRR) channel provides a venue for researchers to publish both confirmatory and non-confirmatory studies to help improve reproducibility of results, mitigate publication bias towards positive results and to promote open dialogue between scientists.
During last month three important announcements were made:
1. The advisory board of PRR was expanded by several important scientists dedicated to improve life sciences;
2. Extensive mislabeling was reported in several reports, most likely only showing the tip of the iceberg;
3. The attempt to replicate the results of the stimulus-triggered acquisition of pluripotency (STAP) protocol published in PRR passed peer review earlier this week.
FacebookXEmailLinkedInWhatsApp