Scientific claims in biomedical research are typically derived from statistical analyses. However, misuse or misunderstanding of statistical procedures and results permeate the biomedical literature, affecting the validity of those claims. One approach journals have taken to address this issue is to enlist expert statistical reviewers. But how many journals actually do this and how is the statistical review incorporated into the editorial process?
To address these questions, Hardwicke and Goodman conducted a survey in 1998 and updated these data in 2020 with the intention of characterizing contemporary statistical review policies at leading biomedical journals. The authors received eligible responses from 107 of 364 (28%) journals surveyed:
34% (36/107) rarely or never use specialized statistical review, 34% (36/107) used it for 10–50% of their articles and 23% used it for all articles. These numbers have changed little since 1998 in spite of dramatically increased concern about research validity. One survey outcome was that the vast majority of editors regarded statistical review as having substantial incremental value beyond regular peer review and expressed comparatively little concern about the potential increase in reviewing time, cost, and difficulty identifying suitable statistical reviewers.
The authors concluded that improved statistical education of researchers and different ways of employing statistical expertise are needed and discussed several proposals for improvement.