Multi-dimensional In-depth Long-term Case studies (MILC)

Description

Follow a few expert users solving their own problems with a complex tool over weeks or months, combining interviews, diaries, logs and outcomes to judge real benefit.

For complex tools such as visual analytics or creativity software, changes in strategy and real benefits appear only after weeks of use, so short lab tasks miss them. MILC studies a small number of experts in their own workplaces, with their own data and goals. A study opens with an interview confirming a real problem, available data and a schedule; then training, two to four weeks of novice use, two to four weeks of mature use, and a semi-structured exit interview. Throughout, diaries, usage logs, questionnaires, observation and outcomes such as decisions or publications give several views. The researcher may help participants and feed tool improvements back during the study, reducing the usual distance to gain depth. Success is judged by whether users reach their own goals, not imposed tasks.

Attributes

Loss type
Siloed
Data
Mixed
Effort
Months

Steps

  1. Recruit a few expert users (often three to six) with real, ongoing problems and data suited to the tool.
  2. Hold an initial interview confirming the problem, the data available, goals and a study schedule.
  3. Run an introductory training session.
  4. Follow two to four weeks of novice use, then two to four weeks of mature use, collecting usage logs, participant diaries, periodic observation and short questionnaires.
  5. Fix or improve the tool as problems surface and record each change and its date.
  6. Close with a semi-structured exit interview on discoveries, benefits, collaboration and frustrations.
  7. Write a case report per user and compare across cases for recurring patterns.

Sources

  • Interaction Design: Beyond Human-Computer Interaction (Preece et al., 2019)
  • Designing the User Interface (Shneiderman et al., 2018)

Also known as

  • MILC
  • MILCs
  • Long-term case studies
  • Multi-dimensional long-term case studies of expert tools
  • Long-term case studies of tool use
  • Longitudinal deployment case study

Categories

  • User Research
  • Field & Ethnographic