Description
When regression errors have unequal variances or are serially correlated, estimate that structure from residuals and refit, giving efficient coefficients and honest standard errors.
Ordinary least squares assumes independent errors with equal variance. Repeated measurements over time on one product metric leave runs of same-sign residuals, and groups (platforms, markets) can differ in noise. OLS coefficients stay consistent but can be several times less efficient, and standard errors can be understated up to threefold. Feasible GLS estimates the error covariance from OLS residuals (an AR(1) coefficient, group variances, an equicorrelation) and refits with it; for AR(1) this is OLS on quasi-differenced data, with Prais-Winsten keeping the first point and Cochrane-Orcutt dropping it. Iterating converges to maximum likelihood, which behaves best in small samples. Kariya and Kurata show the gain over OLS grows with the distance from equal, independent errors, while the cost of estimating the covariance shrinks with residual degrees of freedom.
Attributes
- Discipline
- Statistics & Modelling
- Loss type
- Siloed
- Stage
- Post-launch
- Data
- Quantitative
- Effort
- Days
Steps
- Fit the regression or trend by OLS and save the residuals.
- Diagnose structure: plot residuals over time and against lagged residuals, compute Durbin-Watson (about 2 means none), and inspect the residual ACF and PACF; check spread by group.
- First ask whether the pattern signals a wrong mean model (missing curvature or a break) and revise the regressors if so.
- Choose a covariance structure (AR(1), ARMA, separate group variances, equicorrelation) and estimate its parameters from the residuals.
- Refit by GLS, for AR(1) by OLS on quasi-differenced data, keeping the first case scaled by the square root of 1 minus phi squared.
- Optionally iterate or use maximum likelihood; check that the final residuals are white noise (Ljung-Box or an LM test).
- Report coefficients with GLS standard errors next to the OLS results and the estimated error structure.
Sources
- Generalized Least Squares (Kariya & Kurata, 2004)
- Nonlinear Regression (Seber & Wild, 1989)
- Analysis of Financial Time Series (Tsay, 2002)
Also known as
- Feasible GLS
- FGLS
- Generalised least squares
- Estimated GLS
- Two-stage Aitken estimator
- Regression with AR(1) errors
- Prais-Winsten estimation
- Cochrane-Orcutt procedure
- Regression with ARMA errors
Categories
- Analytics & Behaviour
- Statistics & Modelling