Description
Choose methods and metrics from the research question by answering a few structured questions and reading candidates off a grid, matrix or cheat sheet.
Method selection should start from the question, not a favourite method. Marsh's four questions ask what the research question is, which lifecycle phase applies (define, analyse, evaluate, implement), whether attitudinal or behavioural data are needed, and whether qualitative, quantitative or both; the answers point to methods on a cheat sheet. Albert and Tullis cross ten study scenarios (transactions, product comparison, frequent use, navigation, awareness, problem discovery, critical products, overall experience, subtle changes, alternative designs) with metric types to choose performance, issue, self-report, biometric and live-site metrics. Hinman's grid classes methods by whether the researcher is present and whether the participant is aware of being studied, and asks that a plan cover at least two, preferably three, quadrants so each method's intrusion is offset by another. Together they turn method choice into an explicit, reviewable step.
Attributes
- Discipline
- Research Design & Rigor
- Loss type
- Active
- Goal
- Support a decision
- Stage
- Strategy, Discovery, Evaluation
- Data
- Analytical / expert
- Effort
- Hours
Steps
- Write the research questions and the decisions they inform.
- Identify the lifecycle phase and whether the study is generative, formative or summative.
- Decide whether you need attitudinal or behavioural data, qualitative or quantitative, or both, remembering attitudes usually need larger samples.
- Match the study to a scenario in the Albert and Tullis matrix and read off the recommended metric types.
- Place candidate methods in the presence-by-awareness grid and select a mix covering at least two quadrants.
- Choose tools, sample size and analysis plan for the selected methods and metrics before collecting data.
Sources
- Global UX (Quesenbery & Szuc, 2011)
- User Research (Marsh, 2022)
- Measuring the User Experience (Albert & Tullis, 2022)
- Survey Analysis (Sauro & Lewis, 2021)
- Regression and Key Driver Analysis (Sauro & Lewis, 2021)
- Just Enough Research (Hall, 2013)
- Usability Engineering (Nielsen, 1993)
Also known as
- Presence and Awareness Method Mix
- Research Techniques Framework (Hinman)
- Four-question research method selection with cheat sheet
- Method selection cheat sheet
- UX metric selection by study scenario
- Metric-to-scenario matrix
Categories
- Research Operations
- Research Design & Rigor