Benford's Law Data Integrity Testing

Description

Checks whether naturally occurring numbers in data follow the expected first-digit distribution, flagging fabricated, manipulated or erroneous entries.

In many naturally occurring datasets spanning several orders of magnitude, the leading digit d appears with probability log10(1 + 1/d): about 30% start with 1 and under 5% with 9. Fabricated numbers usually deviate. Forensic accountants use first-digit, first-two-digit and last-digit tests with mean absolute deviation thresholds to flag suspicious records. For UX research and product data, Benford tests can flag fraudulent survey responses (numeric answers), fake reviews with numeric claims, manipulated self-reported metrics, or data pipeline errors. It is a screening tool, not proof, and requires data that should follow Benford.

Attributes

Loss type
Unmigrated
Stage
Evaluation
Effort
Hours

Steps

  1. Confirm the dataset is suitable: naturally occurring, spanning multiple orders of magnitude, not bounded or assigned.
  2. Extract first digits (and optionally first two digits) of the values.
  3. Compare observed with Benford frequencies using MAD with Nigrini's conformity thresholds, plus chi-square for large samples.
  4. Drill into subsets (by respondent, panel source, seller) that deviate most.
  5. Investigate flagged records with other evidence (timing, IP, consistency checks).
  6. Document and exclude confirmed problematic records, reporting the procedure.

Sources

  • Nigrini, Benford's Law: Applications for Forensic Accounting, Auditing, and Fraud Detection (2012)
  • Benford, The law of anomalous numbers (1938), Proceedings of the American Philosophical Society

Also known as

  • Benford analysis
  • First-digit law
  • Digital analysis (forensic accounting)

Categories

  • Information Theory, Physics & Mathematics