Eliciting Dependence Between Uncertain Quantities by Conditional Exceedance Probabilities

Description

Elicit how two uncertain quantities move together by asking experts a conditional probability question instead of asking for a correlation.

Models of uncertain inputs need their dependence as well as their ranges, yet people, statisticians included, cannot state correlations reliably. Kurowicka and Cooke separate two cases. Lumpy dependence arises when one variable switches a process on or off (a market launches or not): elicit the probability of each state and a separate distribution of the other variable for each. Smooth dependence is elicited after both marginal distributions: ask, if Y turned out above its median, what is your probability that X is also above its median? An answer of 1 means complete rank correlation, 0.5 independence and 0 complete negative dependence; intermediate answers convert to a rank correlation through a precomputed chart (0.8 corresponds to about 0.7). With several experts, pool the marginals first, then each expert's probability of joint exceedance of the pooled medians, and convert. Experts quickly become comfortable with the question.

Attributes

Loss type
Siloed
Data
Mixed
Effort
Days

Steps

  1. List pairs of uncertain inputs whose dependence could change the model's conclusions.
  2. For each pair decide whether dependence is lumpy (a switching condition) or smooth.
  3. For lumpy dependence, elicit the probability of each state of the switching variable and a conditional distribution of the other variable for each state.
  4. For smooth dependence, elicit both marginal distributions first, then ask for the probability that X exceeds its median given that Y exceeds its median.
  5. Convert each answer to a rank correlation with the minimum-information conversion chart or software.
  6. With several experts, pool marginals first, compute each expert's probability of jointly exceeding the pooled medians, pool these linearly and convert.
  7. Check that the resulting correlation matrix is positive definite before using it in simulation.

Sources

  • Uncertainty Analysis with High Dimensional Dependence Modelling (Kurowicka & Cooke, 2006)

Also known as

  • Conditional exceedance probability elicitation
  • Probability-of-concordance elicitation
  • Rank correlation elicitation
  • Eliciting dependence between uncertain quantities: conditional exceedance-of-median question and lumpy dependence

Categories

  • User Research
  • Interviewing & Elicitation