Retrospective Practice History Analysis

Description

Reconstructs how people at different skill levels spent their time learning, and how they rate each activity for relevance, effort and enjoyment, to find which activities build expertise.

The method asks people to recall, in yearly intervals, how many hours per week they spent on different practice and play activities, and to rate each activity for its relevance to improving performance, the effort it needs and its enjoyment. Ericsson's work showed that accumulated solitary practice separated elite from less skilled musicians; sport studies repeated the pattern. The Handbook describes the method's weaknesses and remedies: recall inflates estimates, especially for less skilled people, so start with the current year, cross-check against diaries, training logs and time-motion observation of actual sessions, and use quasi-longitudinal comparisons of different age groups. Côté's interview methods add play and the variety of early activities. For UX, the method can be pointed at complex products whose expert users took years to become fluent: spreadsheets, design tools, IDEs, music software, CAD, analytics platforms. Comparing the learning histories of power users and stalled users shows which activities (tutorials, real projects, imitation, community help, playful exploration) actually built skill, which informs onboarding, learning content and in-product practice features.

Attributes

Loss type
Latent
Data
Mixed
Effort
Weeks

Steps

  1. Define skill groups using an external criterion (for example certification level, peer nomination or measured task performance).
  2. List candidate learning activities with users, such as formal courses, tutorials, real projects, exploration, forums and pair work.
  3. Interview or survey each person for weekly hours on each activity by year, starting with the current year and working backwards.
  4. Have them rate each activity for relevance to getting better, effort and enjoyment.
  5. Cross-check a subsample against diaries, product logs or observed sessions to estimate recall bias.
  6. Compare accumulated hours and activity ratings across skill groups, looking for activities that separate them.
  7. Translate distinguishing activities into onboarding, learning content or product features.

Sources

  • Tenenbaum, G. & Eklund, R. (eds), Handbook of Sport Psychology, 3rd ed. (2007)

Also known as

  • Deliberate practice analysis
  • Practice history profiling
  • Retrospective practice questionnaire
  • Career practice estimates

Categories

  • User Research
  • Field & Ethnographic