Description
LLM agents built from generated personas attempt the study task on a real website, and their traces and interviews are used to pilot a usability study before recruiting people.
Planning a usability study involves many guesses about tasks, instructions and which segments will struggle. UXAgent uses LLM agents as simulated participants to test those guesses cheaply. A persona generator produces personas matching a specified demographic distribution from a seed example; each persona drives an agent with a fast loop (perceive, plan, act) and a slow loop (reflect) sharing a memory, and a browser connector lets it act on a live site through simplified page code and generic actions such as click, type and back. Researchers receive action and reasoning traces, outcomes, session recordings and summary statistics, and can interview any agent in a chat loaded with its memory. The authors present it as a way to debug study design and generate hypotheses, not a substitute for real participants; agents' behaviour has not been validated against people.
Attributes
- Discipline
- Usability & Evaluation
- Loss type
- Latent
- Stage
- Evaluation, Design
- Data
- Mixed
- Effort
- Days
Steps
- Define the website and the study task the agents will be given, as you would for a human study.
- Specify the target demographic distribution and an example persona, then generate the required number of personas (for example 60).
- Run one browsing session per persona and collect action traces, reasoning traces, outcomes, recordings and summary statistics.
- Check the data for plausibility first: read a sample of raw traces and discard sessions where the agent misread the page or the task.
- Interview selected agents about their choices, and compare segments to form hypotheses.
- Revise tasks, instructions and screener quotas for the human study, rerun the simulation if needed, and then test the hypotheses with real participants.
Sources
- UXAgent: An LLM-Agent-Based Usability Testing Framework for Web Design (Lu et al., 2025)
Also known as
- UXAgent
- Synthetic-user usability pilot
- LLM agent usability simulation
- Simulated participants for usability testing
Categories
- User Research
- Usability & Evaluation