Description
Updates prior beliefs with observed data to produce posterior distributions, giving direct probability statements and uncertainty estimates useful for small samples and decisions.
Bayesian analysis treats unknown quantities, such as a task success rate or the difference between two designs, as probability distributions. A prior distribution expresses existing knowledge, the likelihood expresses what the data say, and Bayes' theorem combines them into a posterior distribution. This allows direct statements such as 'there is a 92 percent probability that design B has a higher completion rate', credible intervals, decision analyses using expected loss, and natural accumulation of evidence across studies. It is well suited to small UX samples and iterative work. Rigour comes from justifying priors and testing sensitivity to them, checking model convergence and fit, using posterior predictive checks, and avoiding repeated peeking presented as error control.
Attributes
- Discipline
- Statistics & Modelling
- Loss type
- Latent
- Stage
- Evaluation, Post-launch
- Data
- Quantitative
- Effort
- Days
Steps
- Define the quantity of interest and the decision it will inform.
- Specify a prior based on previous data or weakly informative assumptions, and document it.
- Choose a likelihood model appropriate to the data, such as binomial for success rates.
- Compute the posterior analytically or with MCMC software such as Stan, PyMC or JASP.
- Check convergence and model fit with posterior predictive checks.
- Report posterior distributions, credible intervals and probabilities of meaningful effects.
- Test sensitivity to alternative priors and make decisions using expected loss or thresholds agreed in advance.
Sources
- Bayesian Methods for Interaction and Design
- Statistical Rethinking
- Lost Methods in UX — A Complete Treatise (Gopinath)
- Gelman A et al. (2013) Bayesian Data Analysis. 3rd ed
- McElreath R (2020) Statistical Rethinking. 2nd ed
- Kruschke JK (2015) Doing Bayesian Data Analysis. 2nd ed
Also known as
- Bayesian inference
- Bayesian statistics
- Bayesian A/B testing
- Bayesian data analysis
- Credible intervals
- Bayesian Inference for UX Studies
Categories
- Analytics & Behaviour
- Statistics & Modelling