Description
Measure the time from task start to task end under defined clock rules, clean outliers and speeders, and report skew-robust summaries, time bands and threshold attainment.
Task time, the time elapsed between the start and end of a task, matters most for frequently repeated tasks, though faster is not always better in games or learning. Rigour lies in rules fixed before data collection: when the clock starts (participant finishes reading the task) and stops (stops interacting or clicks an answer button), whether only successful tasks count, and how outliers and speeders are removed. Because times are skewed, report the median or geometric mean alongside the mean and confidence interval. Outliers can be screened with mean plus three standard deviations or the Q3 + 1.5 IQR rule; speeders faster than the minimum expert time are removed, and online participants who speed on several tasks are dropped. Further views are time bands and the percentage of participants finishing under a threshold such as twice an expert's time.
Attributes
- Discipline
- Usability & Evaluation
- Loss type
- Active
- Stage
- Evaluation, Post-launch
- Data
- Quantitative
- Effort
- Days
Steps
- Define start and end rules before the study, such as start when the participant finishes reading and end when they click an answer button.
- Collect times with a logger, video timestamps or the testing tool; ask participants to work as quickly and accurately as possible without saying they are timed, and avoid concurrent think-aloud if time is a key measure.
- Decide which tasks count: all tasks if participants chose when to give up, otherwise successful tasks only.
- Screen outliers (above mean + 3 SD, or above Q3 + 1.5 x IQR) and speeders faster than the minimum expert time; drop participants who speed on several tasks.
- Summarise per task with mean and confidence interval plus median or geometric mean; add time bands and the percentage under a threshold such as twice expert time.
- Compare designs with t-tests or ANOVA on log-transformed times.
Sources
- Measuring the User Experience (Albert & Tullis, 2022)
- Five Essential Log Analysis Methods Every UX Researcher Should Know (carousel, n.d.)
- Benchmarking the User Experience (Sauro, 2018)
Also known as
- Time-on-task measurement and analysis
- Task completion time
- Task time analysis
- Completion time outlier and speeder rules
Categories
- User Research
- Usability & Evaluation