Description
Uses prior knowledge from earlier studies to plan experiments that maximise expected information per participant, often with sequential stopping, so fewer participants answer the question.
Bayesian design of experiments chooses sample sizes, conditions and stopping rules by their expected information gain or decision value given a prior distribution, rather than by fixed-power calculations alone. In UX, priors come from past studies, benchmarks or qualitative research. For a task success rate a Beta prior can be updated after each participant, with the study stopping once the posterior is precise enough or the decision is clear. For A/B tests, informed priors and expected-loss stopping rules can shorten experiments. Rigour requires stating and justifying priors, running prior sensitivity analyses, simulating the design's operating characteristics before running it, and reporting posteriors and decision thresholds transparently. McElreath's Statistical Rethinking provides an accessible modelling foundation.
Attributes
- Discipline
- Statistics & Modelling
- Loss type
- Unmigrated
- Stage
- Evaluation
- Data
- Quantitative
- Effort
- Weeks
Steps
- Step 1: Quantify prior from previous studies (what distribution of effect sizes is plausible?).
- Step 2: Set stopping criterion: posterior 90% CI width < 0.10 (for proportions) or < 0.5 SDs (for continuous).
- Step 3: After each batch of N participants, update posterior.
- Stop when criterion is met.
- Compare: Bayesian sequential design typically reduces required N by 20–40% vs. fixed N design.
Sources
- Bayesian Methods in Engineering Design Problems — Laura P. Swiler
- Statistical Rethinking (Richard McElreath)
- Bayesian Methods for Interaction & Design
- Behavioral Drift Playbook (Gopinath)
- Chaloner & Verdinelli, Bayesian experimental design: A review (Statistical Science, 1995)
- Statistical Rethinking (McElreath)
- The Inquiry Series, Ch 40 — Bayesian Thinking & Sequential Study Design
Also known as
- BDoE
- Bayesian optimal design
- Prior-informed study design
Categories
- Analytics & Behaviour
- Statistics & Modelling