A true story of Usability Benchmarking
Katie Taylor at UX Brighton 2018
About this talk
It sounds simple, and it is: usability benchmarking is measuring how many people are successful at a number of tasks in the product or service you’re designing. What happens when there are more things users can do than you have time to benchmark? How do you choose a subset of tasks to measure? Which tasks are most important? And what do you do with your benchmarking results?
Katie talked about her experience of usability benchmarking, using Gerry McGovern’s top tasks method, and adapting it to suit different situations. She warned us about the things to avoid, how to learn from her mistakes, and the benefits of using benchmarking results to inform product design.
About Katie
Katie Taylor is Head of Experience Design at the Wellcome Trust, where she leads a team that works on a diverse range of products from research grants to aids for primary school science teachers. Before joining Wellcome, she worked at the UK’s Government Digital Service (GDS) as lead user researcher on GOV.UK.
Katie has a background in user-centred design and user research. She has previously worked on Ubuntu an open source operating system and designed printers for Hewlett-Packard in Shanghai, China
Related reading
Books, research, ideas and people referenced in this talk.
Ad · Book titles link to Amazon. As an Amazon Associate I earn from qualifying purchases.
Projects and resources
- Wellcome Trust00:25The speaker introduces the Wellcome Trust as a charitable foundation that gives grants to scientists and where she leads experience design.
- Wellcome Collection01:11The talk points to the Wellcome Collection image gallery as a collection of images worth visiting.
- GOV.UK02:20The talk uses GOV.UK as the service in which it ran repeatable benchmark tasks across different teams and devices.
- Check your State Pension forecast02:45The talk uses a state pension task on GOV.UK as an example of a short, repeatable benchmark task.
- Department for Work and Pensions06:36The talk says the pension-content work with DWP raised benchmark performance from about 60 percent to nearly 90 percent.
Concepts
- Usability benchmarking01:42The talk defines usability benchmarking as measuring task success and comparing it again later to create a baseline.
- Top Tasks10:53The talk describes Top Tasks as a method for using user surveys to identify the tasks most important to them before benchmarking.
