Robust Estimation (Trimmed Means, M-Estimators, RANSAC)

Description

Estimate centres, spreads and fits that stay accurate when a few observations are wild: trim the extremes, use the median, down-weight outliers with M-estimators, or fit by consensus.

Robustness means performing well across a range of plausible distributions rather than optimally for one. Messy UX data behave like a contaminated normal: most values well behaved, a fraction from a wide distribution (abandoned tasks, bots, distracted sessions). With 5 percent contamination the median's variance is about 29 percent of the mean's, while being only slightly worse under pure normality. The k-level trimmed mean drops the k smallest and largest values; the median is the extreme case. Huber's M-estimator minimises a loss that is quadratic within a tuning constant k of the centre and linear beyond it, so small k behaves like the median and large k like the mean; the same idea gives robust regression. Simulations favour trimming about 10 to 15 percent at each end, or an M-estimator with k of 1 to 1.5 times a robust scale such as the median absolute deviation. RANSAC fits models to random small subsets and keeps the one with most inliers.

Attributes

Loss type
Latent
Effort
Hours

Steps

  1. Inspect the data for skew and outliers, and examine outliers as possible findings before down-weighting them.
  2. Choose a robust location estimator: a 10 to 15 percent trimmed mean, the median, or a Huber M-estimator with k about 1 to 1.5 times the median absolute deviation.
  3. Estimate spread robustly with the interquartile range or median absolute deviation.
  4. For a fitted relationship, use M-estimation (iteratively reweighted least squares from a good starting fit and scale), or RANSAC when a large share of points are gross outliers.
  5. Compare robust and ordinary estimates; a large gap signals contamination worth investigating.
  6. Report the robust estimate with the estimator, trimming level or tuning constant stated, and use bootstrap intervals where analytic ones are unavailable.

Sources

  • Statistical Inference (Casella & Berger, 2001)
  • Probability and Statistics (DeGroot & Schervish, 2012)
  • Computer Vision: A Modern Approach (Forsyth & Ponce, 2011)
  • The EM Algorithm and Extensions (McLachlan & Krishnan, 1997)
  • Robust Statistics (Huber, 1981)
  • Applied Bayesian Modelling (Congdon, 2003)
  • Multivariate Statistical Inference and Applications (Rencher, 1998)
  • The SAGE Handbook of Quantitative Methods in Psychology (Millsap & Maydeu-Olivares, 2009)
  • Low Sample Size and Regression: A Monte Carlo Approach (Riveros Gavilanes, 2019)
  • Bayesian Statistical Modelling (Congdon, 2006)
  • Nonlinear Regression (Seber & Wild, 1989)
  • Order Statistics (David & Nagaraja, 2003)

Also known as

  • Robust M-estimation (Huber estimator)
  • Huber estimator
  • M-estimators
  • M-estimation
  • Robust regression (M-estimation)
  • Trimmed mean
  • Winsorised and trimmed estimators
  • Median absolute deviation
  • Robust estimation of location and scale
  • RANSAC
  • Random sample consensus
  • Robust covariance and correlation matrices

Categories

  • Analytics & Behaviour
  • Statistics & Modelling