Description
A usability playtest whose focus is chosen by analytics, so the session reproduces a known drop-off or problem and reveals why it happens.
Analytics tell a team where players drop out, fail or stop spending, but not why. An analytics-driven playtest follows the same procedure as a normal observed usability playtest, except that the sections of the game to watch are picked from the data. The narrower the analytics can pin the problem down, for example to one screen or one encounter rather than 'drop-off on Day 3', the better the chance that the issue is reproduced in the lab. Researchers observe players at those points and interview them afterwards, then present the underlying causes and possible fixes, with video of all sessions. Player Research use it in soft launch or after launch, once gameplay is polished, and report that it strengthens understanding and usability insight most, with secondary value for appeal, experience and monetisation (for example why a store or purchase flow underperforms). The method is a simple, structured way to join quantitative and qualitative evidence and applies to any live product with event data.
Attributes
- Discipline
- Usability & Evaluation
- Loss type
- Siloed
- Stage
- Post-launch, Evaluation
- Data
- Mixed
- Effort
- Days
Steps
- Review analytics to locate a specific problem point such as a drop-off, failure spike or underused feature.
- Narrow the location as far as possible (screen, level section, flow step).
- Write a playtest plan and tasks that bring players to that point naturally.
- Recruit target players and run observed one-to-one sessions, recording video.
- Interview each player about what happened at the focus points.
- Synthesise causes, propose fixes, and agree metrics to check whether the fix works after release.
Sources
- Drachen, A., Mirza-Babaei, P. & Nacke, L. (eds), Games User Research (2018)
Also known as
- Telemetry-led playtest
- Data-informed usability playtest
Categories
- User Research
- Usability & Evaluation