n a recent commentary, Dorothy Bishop has referred to “the four horsemen of the reproducibility apocalypse: publication bias, low statistical power, P-value hacking and HARKing (hypothesizing after results are known)” and has indicated that these “problems are older than most junior faculty members”. Do we know how to address these problems?
Indeed, at least for HARKing, “the solution to this problem is quite simple” and was described already back in 1989 by Bishop. Investigators should specify the hypothesis to be tested in advance and specify both sample size and cut-off ranges before collecting the data. If, with hindsight, the data suggest an interesting association that had not been anticipated, a new hypothesis, based on the results, should be formulated, and tested…
So, we do know the problems and we do know what to do to address them. Why do we not see a (faster) change in these questionable research practices? One answer is that the incentives to trigger and sustain such changes are too weak, insufficient, not known or understood.
Two recent examples illustrate how risks to reputation damage could incentivize scientists and organizations to strive for better research practices.

First, in February 2019, scientists associated with the Heidelberg University have announced that they have developed a new test that can accurately detect cancer in the blood. In the coming weeks, this topic was in the news many times – starting with the discussion of who owns the invention, how can this be documented and proven, and ending with an understanding that this was a highly premature claim.

Second, a PLoS Biology publication in April 2019 triggered a re-evaluation of the results that came from the laboratory of one of the most well-known German neuroscientists, Prof. Niels Birbaumer. While the re-evaluation is ongoing (in German only), it is too early to draw any conclusion and we would only express our hopes that, during the study, Prof. Birbaumer had sufficient supervision of how his associate(s) handled the data and the laboratory has proper data documentation practices that will allow a complete and satisfactory reconstruction of the published data. In discussing this story, Patrick Illinger of theSüddeutsche Zeitung wrote, “Die Gefahr liegt auf der Hand, dass Nachwuchsforscher bei der Datenanalyse unter extremem Druck stehen, die vom Professor ersehnte Hypothese experimentell zu bestätigen. Die Folge muss nicht Betrug sein, aber selektive Wahrnehmung liegt nahe.“ (“The danger is obvious that junior researchers are under extreme pressure when analyzing data sets to experimentally confirm the hypothesis desired by his/her professor. The consequence must not be fraud, but selective and biased perception is likely.”).
In the recent LifeSciVC blog post, Deanna Petersen and Paul Newman, cited Warren Buffett saying “It takes twenty years to build a reputation, and five minutes to ruin it,” and added “In biotech, a strong reputation is difficult to build, is even harder to establish quickly, and can be easily tarnished.”
Indeed, for industry, biotech or academia, for individual scientists or organizations, failure to implement and maintain Good Research Practices can lead to consequences damaging reputation. To continue citing Warren Buffett, “If you think about that, you’ll do things differently.”
EQIPD is developing a quality system that is designed to raise scientists’ attention to consequences of not doing things right and to provide scientists with the resources and support to build and implement solutions.