Selectively publishing ‘positive’ results distorts the available evidence for scientific claims. For the past decade, scientists from different fields have been increasingly concerned about the degree of such distortion in their literature. A new publication format, Registered Reports, has been developed to prevent selective reporting. Here, peer review and the decision to publish take place before results are known.
In this article, the authors compared the results in published Registered Reports (N = 71 as of November 2018) with a random sample of hypothesis-testing studies from the standard literature (N = 152) in Psychology. Analysing the first hypothesis of each paper, Anne Scheel and colleagues found 96% positive results in standard reports, but only 44% positive results in Registered Reports. The authors conclude that a plausible factor for this finding is the reduction of publication bias and/or Type-1 error inflation in the Registered-Reports literature.
To our knowledge, this is the first clear evidence that Registered Reports led to a much larger proportion of negative results appearing in the literature and that the amount of experimental outcomes missing from the standard literature may indeed be as large as it was predicted by prior investigations