Maximum Likelihood and Method-of-Moments Estimation

Description

Estimate the parameters of a distribution by matching sample moments or by maximising the likelihood of the observed data, and judge estimators by bias, efficiency, mean square error and consistency.

Point estimation gives a single best value for a model parameter. The method of moments sets sample moments (mean, variance) equal to their theoretical expressions and solves for the parameters; it is simple and gives starting values. Maximum likelihood chooses the parameter values under which the observed data are most probable, usually by maximising the log-likelihood numerically; MLEs are consistent, asymptotically efficient and invariant to transformation, so the MLE of a function of a parameter is that function of the MLE. Estimators are compared by bias, variance, mean square error and consistency. In UX the point is choosing an appropriate distribution for skewed quantities such as task times, session lengths or time-to-churn (for example lognormal or Weibull) and fitting it properly, rather than summarising them with a mean and standard deviation.

Attributes

Loss type
Unmigrated
Effort
Hours

Steps

  1. Choose a candidate distribution that fits the data-generating process (for example lognormal for task times, Weibull for time to event, Poisson for counts).
  2. Compute method-of-moments estimates as a quick check and starting values.
  3. Write the log-likelihood and maximise it numerically to obtain the MLEs.
  4. Obtain standard errors or confidence intervals from the observed information or by bootstrap.
  5. Check fit with a probability plot or by comparing observed and fitted frequencies, and compare candidate distributions with AIC or BIC.
  6. Report the fitted parameters and derived quantities (for example median or 90th percentile) using the invariance property.

Sources

  • Statistical Inference (Casella & Berger, 2001)
  • Probability and Statistics (DeGroot & Schervish, 2012)
  • Mathematical Statistics with Resampling and R (Chihara & Hesterberg, 2011)
  • The EM Algorithm and Extensions (McLachlan & Krishnan, 1997)
  • Latent Variable Models (Loehlin & Beaujean, 2017)
  • Linear Statistical Inference and Its Applications (Rao, 1973)
  • Modes of Parametric Statistical Inference (Geisser, 2006)
  • Univariate Discrete Distributions (Johnson et al., 1992)
  • Loss Models: From Data to Decisions (Klugman et al., 2012)

Also known as

  • Maximum likelihood estimation
  • MLE
  • Method of moments
  • Point estimation by maximum likelihood and method of moments
  • Maximum likelihood estimation and estimator properties

Categories

  • Analytics & Behaviour
  • Statistics & Modelling