Eligibility criteria definition

Description

Defines precisely which users enter an analysis or study, using criteria measurable at time zero, so the population matches the decision and avoids selection bias.

In clinical trials, eligibility criteria define who could be randomised; in target trial emulation they must be applied using only information available at time zero. UX analyses frequently violate this, for example by restricting to users who remained active for 30 days, which conditions on the future and builds survivorship bias into the comparison. Good criteria are explicit, measurable, tied to the population a decision will affect, and documented with a participant flow diagram showing how many were excluded at each step. The same discipline improves recruitment screeners for qualitative studies, where vague criteria produce unrepresentative samples.

Attributes

Loss type
Siloed
Effort
Hours

Steps

  1. State the decision and the population it will affect (e.g. new retail investors in their first 90 days).
  2. Write inclusion and exclusion criteria using only variables known at or before time zero.
  3. Check every criterion for dependence on post-baseline information (activity after time zero, survival, later purchases) and remove such criteria.
  4. Apply criteria sequentially to the data and record counts excluded at each step in a CONSORT-style flow diagram.
  5. Compare included and excluded users on key characteristics to describe generalisability.

Sources

  • Causal Inference: What If (Miguel Hernán & James Robins)
  • DAGs+SEM Playbook (Vivek Gopinath)
  • CONSORT 2010 Statement (Schulz, Altman & Moher)
  • Causal Inference (Hernán & Robins)
  • DAGs+SEM Playbook (Gopinath)
  • The Inquiry Series, Ch 48 — Target Trial Emulation — Causal Evidence from Log Data

Also known as

  • Inclusion and exclusion criteria
  • Study population definition

Categories

  • Research Operations
  • Research Design & Rigor