In 2013, a paper by Katherine S. Button called ‘Power failure: why small sample size undermines the reliability of neuroscience’ was published in Nature Reviews Neuroscience. It got a lot of attention at the time and has since been cited more than 1700 times. The authors concluded that the average statistical power of studies in the neuroscience field is very low. The consequences of this include overestimates of effect size and low reproducibility of results.
Now, four years later, Camilla Nord et al. reanalyzed the same dataset from the original publication and published their finding in the Journal of Neuroscience. The key finding of the new study is that the field of neuroscience is diverse in terms of power, with some branches of neuroscience doing relatively well. The authors demonstrate, using Gaussian mixture modelling, that the sample of 730 studies included in the analysis comprises several subcomponents; therefore, the use of a single summary statistic is insufficient to characterize the nature of the distribution. This indicates that the notion that studies are systematically underpowered is not the full story and low power is far from a universal problem. However, do these findings lessen concerns about statistical power in neuroscience? Unfortunately not. In fact, the authors concluded that the distribution of power is highly heterogeneous demonstrates an undesirable inconsistency, both within and between methodological subfields.