Believed-Cause versus Logged-Data Covariation Check

Description

Elicit exactly which factor people believe drives an outcome and at what delay, then correlate their own logged data at that lag and others to test the belief.

People confidently report causal links that their own records do not support. Redelmeier and Tversky asked 18 arthritis patients which weather variable affected their pain and at what lag, then analysed 15 months of fortnightly measures. The mean correlation at each patient's stated variable and lag was about .02, and across nine lags only 6% of correlations were significant, close to chance, yet nearly all patients believed in a strong effect. A second task found that people correctly identified correlated pairs of random-walk graphs about 90% of the time but saw unrelated pairs as related up to 87% of the time. The mechanism is selective matching: memorable coincidences count, non-matches are ignored, and freedom to pick variables and lags helps. The method pairs belief elicitation with an analysis of the believer's own data.

Attributes

Loss type
Unmigrated
Data
Mixed
Effort
Days

Steps

  1. Elicit the belief precisely: the outcome, the causal factor, the direction and the lag.
  2. Pull the relevant logs for the same people or system covering the outcome and the factor over a long enough period.
  3. Correlate at the stated lag and at a grid of other lags, adjusting for multiple comparisons.
  4. Optionally run a classification task with correlated and uncorrelated series pairs to estimate the group's rate of illusory perception.
  5. Feed both results back to the team or users, noting the limits of coarse logs.

Sources

  • Preference, Belief, and Similarity (Tversky, 2004)
  • On the Belief that Arthritis Pain Is Related to the Weather (Redelmeier & Tversky, 1996)

Also known as

  • Illusory correlation check
  • Belief versus own-data test

Categories

  • Research Operations
  • Research Design & Rigor