Quota-Filling Screening Plan and Stopping Test

Description

Plan how many people to screen to fill a quota in each of two categories with stated confidence, and stop early if a category proves too rare.

Recruiters often need at least c people in each of two groups (users and non-users of a feature, left- and right-handed participants) and screen people one at a time. The number screened until both quotas are filled follows the maximum negative binomial distribution, the larger of two dependent waiting times; its upper tail gives the number of screens that fills both quotas with, say, 95 percent probability. With an even split and 7 needed per group, about 17 screens are expected and 22 give 95 percent assurance. The same distribution supplies a sequential test: if a critical number is reached without filling both quotas, conclude the category proportion is extreme (near 0 or 1) rather than moderate. A without-replacement version applies to small known pools such as a customer panel. Recording the split when screening stops allows the proportion to be estimated.

Attributes

Loss type
Unmigrated
Effort
Hours

Steps

  1. Fix the quota c needed in each of the two categories.
  2. Set a planning value for the proportion p in the first category, or a plausible range (for example 0.1 to 0.9) to serve as the null hypothesis for a test.
  3. From the maximum negative binomial upper tail (or the negative hypergeometric version for a small finite pool), read the number of screens that fills both quotas with the desired probability.
  4. For a stopping test, take the critical number at the boundary of the null range and alpha level; screen sequentially and stop when both quotas are filled or the critical number is reached.
  5. If the critical number is reached first, conclude the category is rarer (or commoner) than assumed and revise the recruitment strategy.
  6. Record how many of each category had been seen when screening stopped, so p can be estimated without ambiguity.

Sources

  • Discrete Distributions: Applications in the Health Sciences (Zelterman, 2004)

Also known as

  • Maximum negative binomial distribution
  • Maximum negative hypergeometric distribution
  • Frequency quotas
  • Noah's ark problem
  • Screen until both types are seen
  • Quota-screening plan and stopping test: trials needed to observe at least c of each of two categories

Categories

  • Research Operations
  • Research Design & Rigor