Goal-Free Evaluation

Description

Evaluate what a product or programme actually does for people, judged against their needs, while the evaluator is deliberately kept ignorant of its stated goals.

In goal-free evaluation the evaluator does not read proposals, objectives or success metrics. Only observable outcomes and documentable effects are studied, and they are judged against assessed participant needs. Scriven's reasons: studying stated objectives narrows attention and misses unanticipated outcomes, the label 'side effects' downgrades important consequences, and knowing the goals biases perception. Qualitative methods suit it because they are open to whatever effects occur. A strong variant runs a goals-based team and a goal-free team in parallel and compares their findings, a form of analyst triangulation. In UX it counters the tendency to evaluate features only against the KPI they were built to move.

Attributes

Loss type
Unmigrated
Effort
Weeks

Steps

  1. Arrange for the evaluator to be screened from goals, proposals, OKRs and success metrics; a gatekeeper filters materials.
  2. Assess the needs of the people affected, independently of the product's intentions.
  3. Observe use and interview users and bystanders to document actual effects, intended or not.
  4. Compare documented effects with assessed needs, rating benefits and harms.
  5. Optionally run a parallel goals-based evaluation and compare the two teams' findings.

Sources

  • Qualitative Research & Evaluation Methods (Patton, 2002)

Also known as

  • GFE

Categories

  • Research Operations
  • Research Design & Rigor