Stochastic Dominance Ranking with a Permutation Test

Description

Decide whether one option's whole outcome distribution beats another's for every decision maker in a broad class, without choosing a utility function, and test the verdict by permutation.

Comparing two designs by their means hides what happens in the tails. Stochastic dominance asks a stronger question: would every person whose preferences meet some mild conditions prefer option A? First-order dominance (A's cumulative distribution lies at or below B's everywhere) covers anyone who prefers more to less; second-order dominance (the running integral of the CDF difference never favours B) adds risk aversion; third and fourth orders add prudence-type conditions and apply to ever narrower classes of people. Anderson's matrix algorithm computes all orders from binned probabilities, and Vinod adds an optional probability-weighting transform for decision makers who depart from expected utility and a permutation test to judge whether an apparent dominance could arise by chance. An indeterminate result is itself informative: the choice then depends on whose preferences count.

Attributes

Loss type
Unmigrated
Effort
Days

Steps

  1. Pool the outcome samples for the two options and divide the common range into class intervals starting just below the minimum.
  2. Compute each option's probability in every interval and cumulate them to empirical CDFs; integrate the CDF differences once, twice or three times for second-, third- and fourth-order comparisons.
  3. Optionally transform the cumulative probabilities with a probability-weighting function to represent people who overweight rare outcomes.
  4. Take the largest difference in each direction as the test statistic for each order.
  5. Build the null distribution by randomly reassigning pooled observations to groups of the original sizes (about 999 permutations) and declare dominance only if the observed statistic reaches the 95th percentile.
  6. Report A over B, B over A or indeterminate at each order; when screening many options against a benchmark, rank by the cumulated dominance vector and check the leaders with bootstrap intervals.

Sources

  • Preparing for the Worst: Incorporating Downside Risk in Stock Market Investments (Vinod & Reagle, 2005)

Also known as

  • Stochastic dominance test
  • 1SD/2SD/3SD/4SD
  • First-order stochastic dominance
  • Second-order stochastic dominance
  • Anderson's stochastic dominance test
  • Distribution dominance comparison
  • Stochastic dominance ranking of alternatives (first to fourth order)

Categories

  • Strategy & Collaboration
  • Game Theory & Decision Science