In light of the reproducibility crisis, there is a growing awareness that the rigorous standardization of experimental conditions may contribute to poor reproducibility of animal studies. Instead, systematic heterogenization has been proposed as a tool to enhance reproducibility, but a real-life test across multiple independent laboratories is still pending.

The aim of this published study was therefore to test whether heterogenization of experimental conditions by using multiple experimenters improves the reproducibility of research findings compared to standardized conditions with only one experimenter. To this end, von Kortzfleisch and colleagues replicated the same animal experiment in 3 independent laboratories, each employing both a heterogenized (i.e., 3 different experiments) and a standardized design (i.e. single experimenter). 

In contrast to the authors’ expectation, the inclusion of multiple experimenters in the heterogenized design did not improve the reproducibility of the results across the 3 laboratories. Interestingly, however, a variance component analysis indicated that the variation introduced by the different experimenters was not as high as the variation introduced by the laboratories, probably explaining why this heterogenization strategy did not bring the anticipated success. Even more interestingly, for the majority of outcome measures, the remaining residual variation was identified as an important source of variance. 

Despite some uncertainty surrounding the estimated numbers, these findings argue for systematically including biological variation rather than eliminating it in animal studies and call for future research on effective improvement strategies.

LINK