Description
Statistically combines effect sizes from multiple studies or experiments, estimating an overall effect, its heterogeneity and moderators.
Organisations run hundreds of studies and experiments but rarely combine them. Meta-analysis converts each study's result into a common effect size with its variance, pools them (fixed or random effects, the latter allowing true effects to vary), and quantifies heterogeneity (tau squared, I squared, prediction intervals). Meta-regression explores moderators like platform or market. Publication bias checks (funnel plots, Egger's test) are directly relevant to internal research too, as unreported null tests bias institutional memory. Within a company, meta-analysis of past A/B tests yields realistic priors and shows whether design patterns generalise.
Attributes
- Discipline
- Research Design & Rigor
- Loss type
- Latent
- Stage
- Strategy
- Data
- Quantitative
- Effort
- Weeks
Steps
- Define the question and inclusion criteria for studies or experiments.
- Extract effect sizes and their standard errors consistently (e.g. log odds ratios, standardised mean differences).
- Fit a random-effects model and report the pooled effect, confidence interval and prediction interval.
- Quantify heterogeneity (I squared, tau) and run meta-regression on planned moderators.
- Assess small-study or publication bias with funnel plots and sensitivity analyses.
- Summarise implications, including when the pattern should not be expected to replicate.
Sources
- Borenstein, Hedges, Higgins & Rothstein, Introduction to Meta-Analysis (2009)
- DerSimonian & Laird, Meta-analysis in clinical trials (1986), Controlled Clinical Trials
- Higgins et al. (eds.), Cochrane Handbook for Systematic Reviews of Interventions (2019)
Also known as
- Meta-analysis
- Random-effects meta-analysis
- Research synthesis (quantitative)
Categories
- Research Operations
- Research Design & Rigor