Bayes Linear Adjustment with Belief Diagnostics

Description

Update a team's stated means, variances and covariances with data by linear fitting, then use diagnostics to see what the data taught and where beliefs and data conflict.

Full Bayesian analysis needs complete joint distributions, which experts can rarely state honestly for many related quantities. Bayes linear analysis takes expectation as the primitive: the analyst elicits only means, variances and covariances for the quantities of interest and the planned observations, and adjusts expectations by the best linear fit on the data, with an adjusted variance as the remaining uncertainty. Resolutions and canonical directions show in advance how informative a planned data source will be, and for which aspects. Once data arrive, standardised changes, discrepancies, the bearing (the direction of greatest change) and the size ratio (observed against expected change, expectation 1) show where beliefs and data conflict, and path correlations show whether different sources agree. Goldstein and Wooff present it as most of the answer for a fraction of the effort.

Attributes

Loss type
Siloed
Data
Mixed
Effort
Days

Steps

  1. Define the quantities to predict and the planned observations, preferably as observable quantities.
  2. Elicit prior means, variances and covariances and check the joint variance matrix is valid (non-negative definite).
  3. Compute adjusted expectations and variances and the resolutions; investigate any resolution of 1, which signals an unintended perfect dependence.
  4. Use partial resolutions to judge which data sources are worth collecting and, under exchangeability, how many observations are needed.
  5. When data arrive, check each standardised change (attention near 2 SD, concern near 3 SD) and the overall discrepancy against its expected value.
  6. Compute the bearing, size and size ratio, adjust by data source in stages to get path correlations, and revise specifications where diagnostics flag conflict.

Sources

  • Bayes Linear Statistics: Theory and Methods (Goldstein & Wooff, 2007)

Also known as

  • Bayes linear analysis
  • Bayes linear methods
  • Adjusted expectation
  • Bayes linear updating
  • Bayes linear belief adjustment

Categories

  • Analytics & Behaviour
  • Statistics & Modelling