Description
Regression for a continuous outcome piled up at a limit, such as many zero spends or a ceiling, or observed only beyond a threshold, where ordinary regression would be biased.
Two data problems look similar but differ. Truncation means cases beyond a limit are never sampled, for example spend recorded only for orders over a minimum. Censoring means those cases are present but recorded at the limit, for example many users with zero spend or ratings stuck at the scale maximum. Simulations show ordinary least squares on such data inflates the intercept and attenuates slopes towards zero. The truncated regression model conditions the normal density on being past the threshold. The Type-1 Tobit treats the limit value as a latent variable at or beyond the threshold and combines a probit-like part for the probability of a non-limit outcome with the density of observed values. A predictor therefore moves the chance of a positive outcome and its size in the same direction. Effects must be read through marginal effects on the probability, the conditional mean and the unconditional mean, not from the latent coefficients alone.
Attributes
- Discipline
- Statistics & Modelling
- Loss type
- Unmigrated
- Stage
- Post-launch, Evaluation
- Data
- Quantitative
- Effort
- Days
Steps
- Inspect the outcome distribution for a spike at a limit (zeros, a ceiling) or a cut-off below which nothing is observed.
- Decide whether the data are truncated (limit cases absent) or censored (limit cases present but uninformative about the latent amount).
- Specify the latent regression with normal errors and the threshold (zero or any constant, from above or below) and estimate by maximum likelihood.
- Check residuals and assumptions: generalised residuals and conditional moment tests for heteroskedasticity and normality, since violations make Tobit estimates inconsistent.
- Compare models with likelihood ratio tests, pseudo-R-squared and AIC or BIC.
- Interpret marginal effects on the probability of exceeding the limit and on the conditional and unconditional expected amount, and validate forecasts on a hold-out sample.
Sources
- Quantitative Models in Marketing Research (Franses & Paap, 2001)
- The EM Algorithm and Extensions (McLachlan & Krishnan, 1997)
Also known as
- Tobit model
- Censored Regression model
- Truncated Regression model
- Limited dependent variable regression
Categories
- Analytics & Behaviour
- Statistics & Modelling