D-prime calculation

Description

Separates a user's true ability to detect a signal (d') from their willingness to respond (criterion), using hit and false-alarm rates.

Signal detection theory models any yes/no detection, such as noticing an alert or spotting a fraudulent transaction, as a decision between overlapping noise and signal-plus-noise distributions. Sensitivity d' = z(hit rate) - z(false alarm rate) measures how far apart those distributions are, independent of response bias; criterion c = -0.5[z(H) + z(F)] captures how liberal or conservative the observer is. This matters because raw accuracy confounds the two: an operator who responds to everything has a perfect hit rate and poor discrimination. In UX, d' tells you whether a design change made signals genuinely more discriminable or merely shifted users' thresholds. Rigour requires a design with both signal and noise trials, enough trials per cell, and corrections for perfect rates.

Attributes

Loss type
Siloed
Stage
Evaluation
Effort
Weeks

Steps

  1. Define the detection task with explicit signal-present and signal-absent trials, for example true alerts versus benign events, in a realistic ratio.
  2. Run each participant through at least 50 signal and 50 noise trials per condition (more if rates are near ceiling) and record hits, misses, false alarms and correct rejections.
  3. Correct rates of 0 or 1 with the log-linear correction (add 0.5 to each cell count and 1 to each trial total).
  4. Compute d' = z(H) - z(F) and criterion c for each participant and condition.
  5. Compare conditions with a within-subjects test on d' (for example paired t-test across 12-20 participants); interpret d' near 1 as weak, 2 as good, and above 3 as excellent discrimination.
  6. Report d' and c separately and recommend design changes for sensitivity (signal salience) versus bias (cost of misses, instructions, alarm credibility).

Sources

  • Van Drongelen, W., Signal Processing for Neuroscientists
  • MacKay, D. J. C., Information Theory, Inference, and Learning Algorithms (2003)
  • Gopinath, V., Forward-Thinking Playbook
  • Van Drongelen (Signal Processing for Neuroscientists)
  • MacKay
  • Forward-Thinking Playbook (Gopinath)
  • The Inquiry Series, Ch 2 — Signal, Noise & the Detection of Intent

Also known as

  • d'
  • Sensitivity index
  • Signal detection sensitivity analysis

Categories

  • User Research
  • Cognitive Psychology & Perception