In vivo problem-solving research

Description

Record real working sessions of experts and code the transcripts for a specific cognitive process, measuring how often and with what consequence it occurs in natural work.

Lab studies of thinking often show processes are rare, while real work may show the opposite. Dunbar's in vivo method recorded real laboratory meetings of molecular biologists and immunologists and coded transcripts for a targeted cognitive process, here the use of analogy to solve problems. Scientists used 3 to 15 analogies per hour-long meeting, and Christensen and Schunn found design engineers developing medical plastic products proposed one about every 5 minutes, with an 'envelope' analogy leading to a paper-based design. This resolved the 'analogical paradox': analogy is rare in lab experiments but common in real work. The method combines naturalistic recording with a precise coding scheme, yielding frequencies, types and consequences of the process, and can be paired with in vitro lab studies to test mechanisms. It is time-consuming and cannot isolate variables.

Attributes

Loss type
Siloed
Stage
Discovery
Data
Mixed
Effort
Months

Steps

  1. Choose the cognitive process of interest (for example analogy, hypothesis testing, error detection) and define it operationally.
  2. Gain access to real working sessions (design reviews, lab meetings, incident calls) and record them with consent.
  3. Transcribe and code each instance of the process with a defined scheme, including its type and source.
  4. Trace the consequence of each instance (did it change the design, solve the problem, get discarded?).
  5. Compute frequencies per hour and by type, and compare across teams or session types.
  6. Where useful, follow up with controlled in vitro studies to test mechanisms suggested by the field data.

Sources

  • Cognitive Psychology (Goldstein, 2019)

Also known as

  • In vivo cognition
  • In vivo/in vitro method

Categories

  • User Research
  • Field & Ethnographic