Realist Evaluation

Description

Explain what works, for whom, in what circumstances and why, by building and testing context-mechanism-outcome configurations rather than average effects.

Realist evaluation rejects the idea that a programme has a single, uniform effect. Programmes work by triggering mechanisms, changes in participants' reasoning and resources, and those mechanisms fire only in certain contexts. The evaluator therefore states candidate programme theories as context-mechanism-outcome (CMO) configurations, collects mixed data on contexts, mechanisms and outcome patterns, and refines the configurations through iterative testing. The 'realist interview' teaches the theory to stakeholders and invites them to confirm or correct it. Findings are expressed as refined middle-range theories that can transfer to other settings, not as a verdict that the programme works. The approach suits qualitative case work and heterogeneous outcomes. Its difficulty is that mechanisms are not directly observable and are easily confused with programme components.

Attributes

Loss type
Siloed
Data
Mixed
Effort
Months

Steps

  1. Elicit initial programme theories from documents and stakeholders, and state them as context-mechanism-outcome configurations.
  2. Decide which contexts, mechanisms and outcomes need data, and choose mixed sources such as logs, surveys, interviews and observation.
  3. Collect data across contrasting contexts and subgroups, using realist interviews to test the theories with stakeholders.
  4. Look for outcome patterns that differ by context and explain them with mechanisms.
  5. Refine, confirm or discard each CMO configuration and repeat with new data as needed.
  6. Report refined configurations stating what works, for whom, in what circumstances and why.

Sources

  • The SAGE Handbook of Qualitative Research (Denzin & Lincoln, 2017)
  • Realistic Evaluation (Pawson & Tilley, 1997)

Also known as

  • Realistic evaluation
  • Context-mechanism-outcome analysis
  • CMO configurations

Categories

  • Research Operations
  • Research Design & Rigor