Multi-Armed Bandit Testing for Adaptive UX

Description

Adaptive experiments that shift traffic toward better-performing variants during the test, using Thompson sampling or UCB to balance learning against serving users the best option.

A fixed A/B test splits traffic evenly until the end, accepting the cost of showing an inferior variant to half of users. Multi-armed bandits treat each variant as an arm with unknown reward and allocate traffic adaptively: Thompson sampling draws from each arm's posterior distribution (for example a Beta distribution for conversion) and serves the arm with the highest draw, while UCB serves the arm with the highest optimistic estimate. Over time traffic concentrates on the best arm, reducing regret. Bandits suit short-lived decisions (headlines, promotions), many variants and personalisation via contextual bandits. The trade-offs are real: adaptive allocation complicates unbiased effect estimation, is vulnerable to time-varying effects and delayed rewards, and is poor for learning why something works. Rigour means choosing bandits for optimisation, not inference, and logging allocation probabilities.

Attributes

Loss type
Unmigrated
Stage
Post-launch
Effort
Weeks

Steps

  1. Implement Thompson sampling or UCB (Upper Confidence Bound) algorithm.
  2. Initialize: equal traffic to all variants.
  3. After each observation: update posterior (Thompson) or confidence bound (UCB).
  4. Shift traffic toward higher-performing variant.
  5. Stop when posterior probability that the best variant is best exceeds 0.95.
  6. Compare total cost (regret) to equivalent fixed A/B test.

Sources

  • Designing with Data — O'Reilly
  • Designing Experiments & Analyzing Data (Maxwell/Delaney)
  • Methods Atlas (Gopinath)
  • Forward-Thinking Playbook (Gopinath)
  • Russo et al., A Tutorial on Thompson Sampling (Foundations and Trends in Machine Learning, 2018)
  • Bandit Algorithms for Website Optimization (John Myles White)
  • The Inquiry Series, Ch 47 — Experimental Design — From A/B to Quasi-Experiments
  • Statistical Rethinking (McElreath)
  • Bayesian Methods for Interaction & Design
  • Behavioral Drift Playbook (Gopinath)
  • The Inquiry Series, Ch 40 — Bayesian Thinking & Sequential Study Design
  • Practical Statistics for Data Scientists (Bruce & Bruce)

Also known as

  • Bandit testing
  • Thompson sampling
  • UCB testing
  • Adaptive experimentation
  • Bayesian bandit
  • Probability matching
  • Posterior sampling allocation

Categories

  • Analytics & Behaviour
  • Experimentation & Causal Inference