Pragmatic (Budget and Feasibility) Sample Sizing

Description

When power analysis or saturation cannot be used, set and justify sample size openly from budget, fixed constraints or published norms, and explain the trade-off to stakeholders.

Power analysis and saturation are the ideals, but budgets, schedules and access usually decide. A cost analysis divides the incentive budget by cost per participant. A feasibility analysis allocates fixed sessions across user groups by importance and population size: with only four focus groups for a hospital discharge study, eight administrators fit in one group, a sample of the 30 clinicians in another, and the large patient population gets two. Heuristics come from local standards and published expert ranges, such as 12 to 20 qualitative interviews, 15 to 30 for card sorts, 6 to 8 per focus group with 3 to 4 groups per participant type, 5 per think-aloud iteration, and 15 to 30 for quantitative usability tests. The rigour lies in being explicit about the basis and what it means for reliability.

Attributes

Loss type
Active
Effort
Hours

Steps

  1. Try power analysis for quantitative work or plan for saturation for qualitative work first.
  2. If neither is feasible, list fixed constraints: budget, time, available sessions, population size, equipment and prototypes.
  3. Compute N from budget (budget divided by cost per participant) or allocate fixed sessions across user groups by importance and population size.
  4. Check the result against local standards and published expert ranges, reading the original sources rather than quoting ranges blindly.
  5. Explain to stakeholders what the limit means for reliability and offer a narrower scope as an alternative.

Sources

  • Understanding Your Users (Baxter et al., 2015)

Also known as

  • Cost, feasibility and heuristic sample sizing
  • Pragmatic sample-size setting
  • Budget-based sample size
  • Feasibility allocation of sessions
  • Local-standards sample size

Categories

  • Research Operations
  • Research Design & Rigor