A poor understanding of what results from statistical tests do and do not mean and an over-reliance on p-values is seen as a key contributor to poor reproducibility of non-clinical studies in the life sciences (Amrhein et al. 2019). Specifically, many investigators do not fully realize how random sampling error can affect study outcomes and result in fickle p-values. Simulations have repeatedly demonstrated how fickle a p-value is (Halsey et al., 2015, Van Calster et al., 2018, Bishop, 2020). However, many not using simulations on a daily basis fail to relate to such findings. Therefore, trainer and trainees from a statistics course for life scientists held at the University of Cologne have developed a tool that enables participants to turn the theoretical knowledge on the fickle p-value into a personal experience (Alawbathani et al., 2021). The tool is based on real data from a published study and allows users to modify various aspects of the sampling and analysis according to their needs. It can be applied in formal courses of statistics, but also as a self-study tool, and is freely available.
Leave A Comment