Direct and Indirect Standardisation

Description

Adjusts rates for differences in population composition (e.g. tenure, device, plan mix) so that groups, markets or periods can be fairly compared.

Comparing raw conversion or churn rates across markets, versions or time periods is misleading if the mix of users differs; this is the root of Simpson's paradox. Direct standardisation applies each group's stratum-specific rates to a common standard population; indirect standardisation applies standard rates to each group's composition to get expected counts, then reports observed/expected ratios (like standardised mortality ratios). Hospitals use case-mix adjustment to compare outcomes. In product analytics this enables fair comparisons of support teams, regions, releases or cohorts with different user mixes, and reveals when apparent improvements are composition shifts.

Attributes

Loss type
Unmigrated
Stage
Post-launch
Effort
Hours

Steps

  1. Choose stratification variables that affect the outcome and differ in distribution (e.g. tenure band, device, plan).
  2. Compute stratum-specific rates for each group being compared.
  3. For direct standardisation, apply rates to a chosen standard population and compute standardised rates.
  4. For indirect standardisation, compute expected events using reference rates and report observed/expected ratios with intervals.
  5. Compare crude and standardised results to show composition effects.
  6. Document the standard population and strata used.

Sources

  • Rothman, Greenland & Lash, Modern Epidemiology (3rd ed., 2008)
  • Preston, Heuveline & Guillot, Demography: Measuring and Modeling Population Processes (2001)

Also known as

  • Age standardisation
  • Standardised rates
  • Standardised ratio (SMR-style)
  • Case-mix adjustment

Categories

  • Epidemiology & Population Health