Ranked-Set Sampling

Description

Rank small random sets by a cheap proxy and measure only one unit per rank in depth, estimating a mean more precisely than simple random sampling.

Some measurements are expensive (an in-depth interview, a full session coding, a lab test) while ranking a handful of units on a cheap proxy is easy. Ranked-set sampling exploits this. Draw k random sets of k units, rank the units within each set by judgement or a cheap auxiliary variable, and measure only the unit ranked first in set one, second in set two, and so on; repeat over m cycles. The mean of the measured units is unbiased even if rankings are imperfect and, with accurate ranking, is up to (k + 1)/2 times as efficient as a simple random sample of the same number of measured units. The gain shrinks as the proxy's correlation with the target falls. Set sizes are kept small, typically 3 to 5, so ranking stays reliable. Variants measure only set medians for symmetric populations.

Attributes

Loss type
Siloed
Data
Mixed
Effort
Days

Steps

  1. Identify the costly measurement and a cheap proxy that should correlate with it (for example usage minutes as a proxy for task proficiency).
  2. Choose a set size k (3 to 5) and number of cycles m.
  3. Draw m times k squared units at random and split them into sets of k.
  4. Rank units within each set on the proxy or by judgement, without the costly measurement.
  5. In the i-th set of each cycle, take the costly measurement only on the unit ranked i.
  6. Estimate the mean as the average of measured units, estimate its variance from the cycle structure, and compare efficiency with simple random sampling given the proxy correlation.

Sources

  • Order Statistics (David & Nagaraja, 2003)

Also known as

  • RSS
  • McIntyre ranked-set sampling
  • Judgement-ranked sampling
  • Double sampling with concomitants

Categories

  • Research Operations
  • Research Design & Rigor