Description
Rating scales whose points are defined by concrete behavioural examples derived from critical incidents, improving rater agreement and meaning.
BARS were developed for performance appraisal. Critical incidents of effective and ineffective behaviour are collected from experts, grouped into dimensions, retranslated by a second group to confirm dimension membership, then scaled by experts so each incident anchors a scale point. Raters then judge observed behaviour against concrete descriptions rather than vague adjectives. Agreement improves and ratings are more defensible. For UX, BARS suit observer ratings of task performance, severity of usability problems, quality of AI outputs, or rubric-based design critique, where adjectives like 'good' and 'poor' produce inconsistent ratings.
Attributes
- Discipline
- Survey & Psychometrics
- Loss type
- Unmigrated
- Stage
- Evaluation
- Data
- Mixed
- Effort
- Weeks
Steps
- Collect critical incidents of good and poor performance or quality from experts (e.g. 50-200).
- Group incidents into dimensions and define each dimension.
- Retranslation: have a second group assign incidents to dimensions; keep those with at least 60-80% agreement.
- Have experts rate each retained incident's level; keep incidents with low variance as anchors.
- Build scales with behavioural anchors at key points and train raters with examples.
- Measure inter-rater reliability in a pilot and revise anchors with poor agreement.
Sources
- Smith & Kendall, Retranslation of expectations (1963), Journal of Applied Psychology
- Schwab, Heneman & DeCotiis, Behaviorally anchored rating scales: A review of the literature (1975), Personnel Psychology
Also known as
- BARS
- Behaviourally anchored scales
- Smith-Kendall scales
Categories
- Analytics & Behaviour
- Survey & Psychometrics