Description
Set and track a minimum amount of user research each team member observes, and score how engaged product teams are during studies.
Teams that watch users make better decisions, so this method measures exposure directly. At GOV.UK everyone on a team had to observe at least two hours of user research every six weeks; 'have you done your two hours?' became common currency, and a public dashboard showed sessions run and the share of the team who had observed one recently. Albert and Rosenberg's Team Engagement Score combines three ratios into a 0 to 1 score: commitment (sessions observed over sessions observable), prioritisation (sessions observed in person over sessions observable in person) and involvement (the moderator's rating of the team's questions and feedback). Tracked across studies and teams, the measures show where research is being consumed and where it is being ignored.
Attributes
- Discipline
- Research Design & Rigor
- Loss type
- Latent
- Stage
- Post-launch, Strategy
- Data
- Quantitative
- Effort
- Hours
Steps
- Set an exposure quota, for example at least two hours of observed research per person every six weeks.
- Log who observes each session, and whether in person or remotely.
- For each study compute commitment (sessions observed / observable) and prioritisation (in-person observed / observable in person).
- Have the moderator rate team involvement during and after sessions on a fixed scale.
- Combine into a 0-1 engagement score and publish exposure and engagement on a simple dashboard.
- Review trends with team leads and follow up with teams that fall below the quota.
Sources
- Measuring the User Experience (Albert & Tullis, 2022)
- Designing with Data (King et al., 2017)
Also known as
- Team Engagement Score (TES)
- TES
- Research exposure quota with a public dashboard
- Exposure hours
- Two hours every six weeks
- UX maturity metric for observation
Categories
- Research Operations
- Research Design & Rigor