Staged-World Field Studies with Cognitive Process Tracing

Description

Study how teams handle demanding work by staging realistic simulated scenarios with inserted probes, then tracing their cognitive process through layered protocols compared across teams and scenarios.

Woods places staged worlds between natural observation, where critical events are rare, and spartan laboratory tasks, which strip away what matters. The researcher models what matters in the field, designs scenarios with an explicit mapping between the target situation and the test, and inserts probes such as complicating events or new prototypes, which serve as instruments of discovery rather than products under test. Data from several sources are collated into a base protocol per team, then re-described in the study's concepts as a formal protocol that keeps interpretation separate from observation. Protocols are contrasted across teams, probes and scenarios, and integrated with other studies into functional accounts of how the system copes. This process tracing is the cognitive engineering method, not the political-science technique of the same name.

Attributes

Loss type
Siloed
Effort
Months

Steps

  1. Define the cognitive-system questions (for example how anomalies are noticed or how work is handed over), avoiding superficial labels.
  2. Map the target situation to the test: problem space, scenario design and which situations to sample.
  3. Build the scenario with artefacts and procedures that externalise cognitive activity, and insert probes such as complicating events or a prototype.
  4. Map canonical expected behaviour so surprises can be recognised.
  5. Run scenarios with teams, collecting several data sources (video, logs, talk, artefacts).
  6. Collate a base protocol per team, then re-describe it as a formal protocol in the study's concepts.
  7. Contrast protocols across teams, probes and scenarios, and integrate with converging studies into a functional account.

Sources

  • Handbook of Cognitive Task Design (Hollnagel, 2003)

Also known as

  • Staged-world field studies with process tracing
  • Staged worlds
  • Scenario-based observation with probes

Categories

  • User Research
  • Field & Ethnographic