Sample Size Matters – An App Exploring Sample Size and Power

Problems with statistical analysis and data visualization in published papers (REF1REF2REF3REF4) have led to calls to improve training for biomedical scientists. Although statistics are essential for biomedical research, statistics training is not always required to complete a PhD.
A new set of free online simulators allow readers visually to explore statistical concepts. Each simulator includes a series of questions. Users can adjust sample sizes and other parameters to explore concepts and find answers. Sample questions include:

  • Distributions and summary statistics: How many data points do you need to determine whether the data are normally distributed?
  • Power and p-values: How likely is it that you will get a significant p-value if your hypothesis is true, and the power for your study is 50%? How likely is it that you will get a significant p-value if your hypothesis is false and there is no effect?
  • Effect sizes: Why doesn’t a smaller p-value mean that you’ve found a larger effect? Why do underpowered studies that find significant differences tend to overestimate the size of the effect?
  • Publication bias: Why is publication bias a greater problem for small effects, compared to large effects? Why is publication bias more likely to occur in fields where researchers conduct small, underpowered studies?

Scientists can use the simulators independently, or integrate them into courses or laboratory training. The simulators are also included in an online course: “Sample Size Matters: Misconceptions about Graphs and Statistical Analyses in Lab and Clinical Research”. Each unit examines misconceptions about data visualization and statistical analyses that are common in fields that regularly perform small sample size studies. Examples of misconceptions include “bar graphs can be used to present continuous data”, “a smaller p-value means you’ve found a larger effect”, or “being underpowered only matters if you don’t find a significant difference”. The course is self-paced, takes approximately ten hours to complete and costs $50. Scientists can find more information or register HERE.

Share this

Leave a Reply

Your email address will not be published. Required fields are marked *