Description
Model-based estimates for subgroups too small for direct survey estimates, from a multilevel model fitted to the survey and projected onto known population counts.
Direct survey estimates for small domains (a region, a segment within a market, a plan tier in one country) are often too noisy to use. Small area estimation fits a multilevel model in which domain or cluster effects are partly pooled, with covariates at both levels, so thin domains borrow strength from similar ones. Known population counts for every cross-class of the model's categories then let the analyst predict outcomes for the unsampled members of each cell from the posterior predictive distribution, add the observed sample outcomes and aggregate to any domain of interest, with credible intervals. Congdon's example pooled 33 states with about 64 effective parameters in place of 103 fixed effects and reduced mean absolute prediction error from about 7.2 to 5.3. The method stands or falls on the accuracy of the population counts and on model fit.
Attributes
- Discipline
- Survey & Psychometrics
- Loss type
- Siloed
- Stage
- Discovery, Post-launch
- Data
- Quantitative
- Effort
- Weeks
Steps
- Define domains as cross-classes of characteristics whose population counts are known (for example plan, region and tenure band).
- Fit a multilevel model (for example a logit for a yes/no outcome) with random domain or cluster effects and cluster-level covariates.
- Obtain population counts for every cell, from the customer base, CRM or census.
- Predict outcomes for the unsampled members of each cell from the posterior predictive distribution and add the observed sample outcomes.
- Aggregate to the domains of interest and report estimates with credible intervals.
- Compare with direct estimates where samples are large, and check model fit before trusting the thinnest domains.
Sources
- Applied Bayesian Modelling (Congdon, 2003)
Also known as
- Small area estimation
- Small domain estimation
- SAE
- Multilevel regression and post-stratification
- MRP
- Model-based domain estimation
Categories
- Analytics & Behaviour
- Survey & Psychometrics