UX KPIs with Baselines and Pre-Set Success Thresholds

Description

Defines goal-linked UX indicators with explicit formulas, measures baselines before change, fixes success and failure thresholds in advance, and reports a few rates regularly.

The sources converge on one discipline. Start from intent and break it into goals with a baseline and a measure of progress; choose a few action-oriented indicators tied to business goals, stated as rates rather than counts. Wendel adds paired metrics: an outcome metric for the real-world result and an action metric for the behaviour that drives it, each written as a formula (what, how, over what period) and checked for being accurate, reliable, rapid, responsive, sensitive and cheap, preferring automatic tracking over self-report. Baselines are measured before change, since without them assumptions mislead. Success and failure thresholds, the 'line in the sand', are fixed before results arrive, with a decision written for each outcome, guarding against post-hoc spin; the same minimum effect sets sample sizes and stopping rules. Indicators are reported on a regular rhythm, typically a one-page monthly report with context and interpretation, and flags deliver data when something important happens.

Attributes

Loss type
Vestigial
Effort
Days

Steps

  1. Restate the intended outcome and break it into specific goals; check each is tangible and able to show both success and failure.
  2. Write an outcome metric and an action metric as explicit formulas (what, how, over what period), stated as rates rather than counts.
  3. Check each metric for accuracy, reliability, speed, responsiveness, sensitivity and cost; avoid self-reported actions where tracking exists.
  4. Measure baselines before any change, over a stated baseline period.
  5. Set success and failure thresholds before measuring, from the business model, benchmarks or a trend asymptote, and write down the decision for each result; record and explain any later change.
  6. Set flags for situations that should trigger notification, such as a sudden drop in task completion.
  7. Report a handful of rates monthly on one page with month-on-month and year-on-year change, interpretation and follow-up, including 'all is well' or 'under watch'.

Sources

  • How to Make Sense of Any Mess (Covert, 2014)
  • Universal Methods of Design (Martin & Hanington, 2019)
  • Designing for Behavior Change (Wendel, 2014)
  • Lean Analytics (Croll & Yoskovitz, 2013)
  • Practical Web Analytics for User Experience (Beasley, 2013)
  • User Experience Management (Lund, 2011)
  • An Elegant Puzzle (Larson, 2019)

Also known as

  • UX key performance indicators
  • Goals, indicators, baselines and flags
  • Outcome and action metrics with pre-committed success thresholds
  • Line in the sand
  • Success and failure thresholds
  • Regular UX analytics reporting
  • UX KPI report

Categories

  • Analytics & Behaviour
  • Product Analytics & Strategy