Description
Flag single bad readings in a metric series and fill missing values, using a time-series model to judge how implausible each point is.
An additive outlier is one recorded value shifted by an unknown amount, such as a logging glitch or a double-counted batch; it biases estimates and can mislead model choice. The idea is to treat a suspect point as missing: its conditional distribution given the model and its neighbours shows how unlikely the recorded value is. McCulloch and Tsay model the series as an outlier-free autoregression plus, at each time point, an indicator that is switched on with small probability and an outlier of random size. Gibbs sampling draws the autoregressive coefficients, variance, indicators, sizes and outlier rate in turn, and draws missing values from their conditional distributions. Each time point gets a posterior probability of being an outlier and an estimated size. The joint approach is slower than sequential detection but less prone to spurious flags when several outliers are present.
Attributes
- Discipline
- Statistics & Modelling
- Loss type
- Unmigrated
- Stage
- Post-launch
- Data
- Quantitative
- Effort
- Days
Steps
- Choose an autoregressive order for the clean series from plots and partial autocorrelations.
- Model each observation as the clean series plus an indicator (prior outlier rate, for example 5 percent) times an outlier size.
- Set conjugate priors: normal for AR coefficients and outlier sizes, inverse chi-squared for the variance, beta for the outlier rate.
- Run a Gibbs sampler (for example 1,000 kept iterations after a burn-in), drawing coefficients, variance, indicators, sizes and missing values in turn, and check convergence.
- Report each time point's posterior outlier probability and size, and the imputed values for gaps.
- Investigate flagged points against deployment and logging records before correcting or excluding them, watching for patches of similar outliers that can mask each other.
Sources
- Analysis of Financial Time Series (Tsay, 2002)
Also known as
- Bayesian time-series outlier detection
- Additive outlier detection
- Gibbs sampling outlier detection
- Additive outlier detection and missing-value imputation in a time series by Gibbs sampling
Categories
- Analytics & Behaviour
- Statistics & Modelling