Description
Estimates success rates, such as task completion or conversion, as a Beta distribution updated with observed successes and failures, giving full uncertainty with simple arithmetic.
For binary outcomes the Beta distribution is the conjugate prior of the binomial likelihood: if the prior is Beta(a, b) and you observe s successes and f failures, the posterior is Beta(a + s, b + f). This makes Bayesian estimation of completion rates, conversion rates or error rates trivial to compute and easy to explain. The posterior gives credible intervals and direct probability statements, such as the probability that the completion rate exceeds 80%, or that variant B beats A. Small-sample usability studies benefit especially, because the posterior remains well behaved at 0 or 100% success, where Wald intervals fail. Rigour comes from stating and justifying the prior, checking sensitivity to it, and using posterior simulation for comparisons.
Attributes
- Discipline
- Statistics & Modelling
- Loss type
- Latent
- Stage
- Evaluation
- Data
- Quantitative
- Effort
- Hours
Steps
- Define the binary outcome precisely, such as task success under defined criteria or conversion within a session.
- Choose a prior: Beta(1, 1) for uniform ignorance, Beta(0.5, 0.5) for the Jeffreys prior, or an informative Beta based on past studies with prior strength equivalent to a stated number of observations.
- Update with data: posterior is Beta(a + successes, b + failures).
- Report the posterior mean and 95% credible interval, and answer decision questions directly, such as P(rate > 0.8).
- To compare variants, draw 10,000 or more samples from each posterior and compute P(B > A) and the distribution of the difference or ratio.
- Check prior sensitivity by repeating with a weak prior and report whether conclusions change.
Sources
- Statistical Rethinking (McElreath)
- Bayesian Methods for Interaction & Design
- Behavioral Drift Playbook (Gopinath)
- The Inquiry Series, Ch 40 — Bayesian Thinking & Sequential Study Design
Also known as
- Beta-Binomial model
- Bayesian proportion estimation
- Beta posterior for conversion rates
Categories
- Analytics & Behaviour
- Statistics & Modelling