Probability Sampling

Description

Choosing survey or study participants by a chance process from a defined population so that results can be generalised with a known margin of error.

Probability sampling gives every member of a defined population a known, non-zero chance of selection. Gratton and Jones set out four variants. Simple random sampling gives everyone an equal chance, using a draw, a random number table or software. Stratified random sampling divides the population into subgroups (for example by age or sex) and draws randomly within each, so that each subgroup is properly represented. Cluster sampling randomly selects whole groups (such as teams or sites) and studies everyone in them. Systematic sampling takes every kth case from a list or a stream of people, which works only when the list order is not itself patterned. The book stresses that the first step is defining the population, that sample size, not the percentage of the population, drives precision, and that every subgroup to be compared needs enough cases. For UX researchers, most participant pools are convenience or panel samples, so claims such as '40% of users want X' are often stated with no basis for generalisation. Drawing an intercept or email sample at random from the real user base, stratified by plan or platform, is cheap with product data and makes survey percentages defensible.

Attributes

Loss type
Siloed
Effort
Days

Steps

  1. Define the population precisely (for example all accounts active in the last 90 days) and obtain a sampling frame such as a user list.
  2. Decide which subgroups must be compared and whether to stratify on them (plan tier, platform, region).
  3. Choose the variant: simple random, stratified random, cluster (random selection of teams, sites or organisations) or systematic every kth case.
  4. Set the target achieved sample size from the precision you need overall and within each subgroup, allowing for expected non-response.
  5. Draw the sample with software or a random number generator and log the seed and frame date.
  6. Field the study, then compare the achieved sample against the frame to check representativeness before generalising.

Sources

  • Gratton, C. & Jones, I., Research Methods for Sports Studies (2010)
  • Veal, A. J. & Darcy, S., Research Methods in Sport Studies and Sport Management (2014)

Also known as

  • Random sampling
  • Simple random sampling
  • Stratified random sampling
  • Cluster sampling
  • Systematic sampling (every kth case)
  • Multi-stage sampling
  • Probability proportional to size sampling

Categories

  • Research Operations
  • Research Design & Rigor