Null-hypothesis significance testing (NHST) is commonly used in psychology and biomedical research areas. However, statisticians have long warned that researchers regularly misinterpret and misapply the technique.
In this paper, published by Scott Cassidy and colleagues, the authors investigated whether introduction-to-psychology textbooks actually accurately define and explain statistical significance.
In an analysis of 30 introductory psychology textbooks, it was found that the vast majority of these books defined or explained statistical significance inaccurately.
The most common misconception identified was the odds-against-chance fallacy, defining statistical significance as a less than a 5% likelihood of a result being random.
The authors conclude: “Overall, these results suggest that students’ misinterpretations of statistical significance may not be the result of their failing to remember the correct interpretation they were taught,” the scientists write. “Instead, students may be accurately recalling incorrect pedagogy.”

LINK