Meta-analysis is a complex statistical method which involves synthesis of data from relevant studies to devise an effect size or a conclusion and has increasingly been recognized as an important tool in the field of biomedical life sciences. Meta-analyses often include an assessment of publication bias based on visual or analytical detection of asymmetry in funnel plots.
In this article, published in eLIFE, Peter-Paul Zwetsloot and colleagues studied the influence of different normalisation approaches, sample size and intervention effects on funnel plot asymmetry, using empirical datasets and illustrative simulations. The authors found that funnel plots of the Standardized Mean Difference (SMD) plotted against the standard error (SE) are susceptible to distortion, leading to overestimation of the existence and extent of publication bias. Distortion was more severe when the primary studies had a small sample size and when an intervention effect was present. Furthermore, the authors showed that using the Normalised Mean Difference measure of effect size or plotting the SMD against a sample size-based precision estimate, are more reliable alternatives and concluded that funnel plots using the SMD in combination with the SE are unsuitable for publication bias assessments as it can lead to false positive results.