User-Oriented Evaluation

Description

Evaluate a service through the eyes of its intended beneficiaries, deriving the evaluation criteria bottom-up from their needs, values and experiences.

A Scandinavian evaluation model for public services that starts from the people the service is meant to benefit. It is explicitly not a satisfaction survey, where the provider sets the topics and users merely score them. Instead, qualitative inquiry finds the issues users raise in their own language, and these become the criteria against which the service is assessed. The model names its own tensions: users who cannot speak for themselves, such as people with dementia; preferences that conflict with wellbeing; conflicting needs between different users; paternalism returning when evaluators speak on users' behalf; and drift into consumerism. Handling these explicitly is part of its rigour. It sits close to responsive evaluation but focuses specifically on beneficiaries rather than on all stakeholders.

Attributes

Loss type
Siloed
Data
Mixed
Effort
Weeks

Steps

  1. Define the intended beneficiaries of the service and distinguish them from other stakeholders.
  2. Explore their experiences qualitatively in their own language through interviews, observation or proxies where needed.
  3. Derive evaluation criteria from the concerns they raise, not from provider objectives.
  4. Assess the service against those criteria using mixed evidence.
  5. Make explicit how conflicting needs between users and users who cannot articulate their needs were handled.
  6. Report criteria, assessment and the trade-offs made.

Sources

  • The SAGE Handbook of Qualitative Research (Denzin & Lincoln, 2017)

Also known as

  • Client-oriented evaluation

Categories

  • Research Operations
  • Research Design & Rigor