Description
Selects each next question from a calibrated item bank to be most informative at the respondent's current estimated level, stopping at a fixed length or target precision.
A fixed questionnaire wastes items that are far from a respondent's level: people with very high satisfaction learn little from items that only discriminate at the low end. A computerised adaptive test needs five components: a calibrated IRT item bank, a starting rule, an item selection rule, a stopping rule and a scoring method. Item information is additive and roughly the inverse of the squared standard error, so precision at a level depends on how many items target it. Maximum-information selection picks the item most informative at the provisional estimate; before responses vary, MAP or EAP estimation with a prior keeps that estimate finite. Fixed-length stopping gives uniform length; precision-based stopping gives shorter questionnaires for most people and suits cut-score decisions, but is limited by bank coverage. Health outcome systems such as PROMIS use CAT to measure constructs precisely with four to eight items.
Attributes
- Discipline
- Survey & Psychometrics
- Loss type
- Unmigrated
- Stage
- Evaluation, Post-launch
- Data
- Quantitative
- Effort
- Months
Steps
- Build and calibrate an item bank with an IRT model on a large, representative sample (typically several hundred to over a thousand respondents), checking unidimensionality and fit.
- Choose a starting item or starting estimate (for example the population mean).
- After each response, update the provisional trait estimate with EAP or MAP.
- Select the next item with maximum information at that estimate, subject to any content-balance or exposure constraints.
- Stop at a fixed number of items or when the standard error falls below a target (for example 0.3 on the theta scale).
- Report the final IRT score and standard error, and monitor item exposure and bank drift over time.
Sources
- The SAGE Handbook of Quantitative Methods in Psychology (Millsap & Maydeu-Olivares, 2009)
Also known as
- Tailored testing
- Adaptive questionnaire
- IRT-based adaptive survey
- Computerized adaptive testing
Categories
- Analytics & Behaviour
- Survey & Psychometrics