Description
Rank small random sets by a cheap proxy and measure only one unit per rank in depth, estimating a mean more precisely than simple random sampling.
Some measurements are expensive (an in-depth interview, a full session coding, a lab test) while ranking a handful of units on a cheap proxy is easy. Ranked-set sampling exploits this. Draw k random sets of k units, rank the units within each set by judgement or a cheap auxiliary variable, and measure only the unit ranked first in set one, second in set two, and so on; repeat over m cycles. The mean of the measured units is unbiased even if rankings are imperfect and, with accurate ranking, is up to (k + 1)/2 times as efficient as a simple random sample of the same number of measured units. The gain shrinks as the proxy's correlation with the target falls. Set sizes are kept small, typically 3 to 5, so ranking stays reliable. Variants measure only set medians for symmetric populations.
Attributes
- Discipline
- Research Design & Rigor
- Loss type
- Siloed
- Stage
- Evaluation, Discovery
- Data
- Mixed
- Effort
- Days
Steps
- Identify the costly measurement and a cheap proxy that should correlate with it (for example usage minutes as a proxy for task proficiency).
- Choose a set size k (3 to 5) and number of cycles m.
- Draw m times k squared units at random and split them into sets of k.
- Rank units within each set on the proxy or by judgement, without the costly measurement.
- In the i-th set of each cycle, take the costly measurement only on the unit ranked i.
- Estimate the mean as the average of measured units, estimate its variance from the cycle structure, and compare efficiency with simple random sampling given the proxy correlation.
Sources
- Order Statistics (David & Nagaraja, 2003)
Also known as
- RSS
- McIntyre ranked-set sampling
- Judgement-ranked sampling
- Double sampling with concomitants
Categories
- Research Operations
- Research Design & Rigor