Description
Freezes a simulation at random points, blanks the display and asks users queries about what is happening, objectively measuring perception, comprehension and projection.
Endsley defined situation awareness at three levels: perception of elements (Level 1), comprehension of their meaning (Level 2) and projection of future status (Level 3). SAGAT measures it objectively by pausing a realistic simulation at unpredictable moments, hiding all displays and administering queries derived from a goal-directed task analysis of SA requirements. Answers are scored against ground truth from the simulation. Because the freezes are random and queries numerous, users cannot anticipate them, and the scores reflect what they genuinely hold in mind. SAGAT reveals whether a dashboard actually supports understanding and anticipation, which preference ratings and task completion cannot.
Attributes
- Discipline
- Human Factors & Error
- Loss type
- Siloed
- Stage
- Evaluation
- Data
- Quantitative
- Effort
- Weeks
Steps
- Conduct a goal-directed task analysis to list SA requirements at levels 1, 2 and 3 for the user's goals.
- Build a realistic, scripted simulation or high-fidelity prototype with ground truth for every queried element.
- Plan 3-6 freezes per session at random times, avoiding the first 3-5 minutes, each lasting up to about 2 minutes.
- At each freeze, blank the display and administer a random subset of queries; record answers.
- Score answers against ground truth per level and per requirement; compare designs with within-subject designs where possible.
- Identify requirements with low accuracy and redesign displays to support them; retest.
Sources
- Endsley, Toward a theory of situation awareness in dynamic systems (1995), Human Factors
- Endsley, Measurement of situation awareness in dynamic systems (1995), Human Factors
- Endsley & Jones, Designing for Situation Awareness (2nd ed., 2011)
Also known as
- SAGAT
- Situation Awareness Global Assessment Technique
- Freeze-probe technique
Categories
- Accessibility
- Human Factors & Error