Sample size planning (G*Power)

Description

Uses G*Power or equivalent tools to calculate the sample needed for a specific test, effect size, alpha and power before running a quantitative UX study.

G*Power is a free tool that computes required sample sizes, achievable power and detectable effects for t-tests, F-tests (ANOVA, regression), chi-square tests, z-tests and exact tests. It is the standard tool in psychology and behavioural science for a priori power analysis. Applied to UX it forces precise decisions about design (between or within subjects), the test family, the expected correlation among repeated measures, and the effect size of interest. Within-subjects designs, common in usability comparisons, can require far fewer participants when measures are correlated, and G*Power makes this explicit. Rigour comes from justifying each input, entering the effect size in the metric the tool expects (for example f for ANOVA, not d), and documenting the screenshot or parameters in the study plan.

Attributes

Loss type
Siloed
Effort
Hours

Steps

  1. Select the test family and statistical test matching the analysis plan (for example t-tests, means: difference between two dependent means for a within-subjects comparison).
  2. Choose 'A priori: compute required sample size' and enter the tail(s), alpha (for example 0.05) and power (for example 0.80).
  3. Enter the effect size in the required metric (d, f, w or f²), derived from pilot data, prior studies or the smallest effect of interest, using G*Power's effect size calculator if needed.
  4. For repeated measures, enter a realistic correlation among measures (often 0.5 to 0.7 for task ratings), and a non-sphericity correction if relevant.
  5. Record the output sample size, inflate for expected exclusions or attrition (for example 10% to 20%), and round up to balance conditions.
  6. Save the protocol output and include it in the pre-registration or research plan.

Sources

  • Statistics for Dummies series (Rumsey)
  • Designing Experiments & Analyzing Data (Maxwell & Delaney)
  • The Inquiry Series, Ch 41 — Frequentist Inference Done Right
  • Companion Book to Quantifying the User Experience (Lewis & Sauro)
  • Benchmarking the User Experience (Sauro)
  • Quantifying the User Experience (Sauro & Lewis)

Also known as

  • G*Power analysis
  • Sample size calculation
  • A priori sample size estimation

Categories

  • Research Operations
  • Research Design & Rigor