Description
A short qualitative pre-study that judges whether a programme or product initiative is defined and implemented clearly enough for a formal evaluation to be meaningful.
Before spending on outcome evaluation, the assessor uses interviews, document review and observation to check three things: that the treatment (the intervention, feature or service) is identifiable and logical, that intended outcomes are clear, specific and measurable, and that what is actually implemented links plausibly to those outcomes. If any fails, a formal evaluation would measure noise. Patton notes that the process often clarifies the logic model and frequently changes the programme before any outcome evaluation is done, which is a benefit in itself. In UX it is the missing gate before an expensive impact study or A/B test of a loosely defined initiative.
Attributes
- Discipline
- Research Design & Rigor
- Loss type
- Unmigrated
- Stage
- Strategy, Evaluation
- Data
- Qualitative
- Effort
- Days
Steps
- Interview managers and staff about goals, activities and expected outcomes.
- Review documents such as proposals, roadmaps, specifications and success metrics.
- Observe the initiative in operation to see what is actually delivered.
- Judge whether the treatment is identifiable, outcomes are specific and measurable, and implementation links logically to outcomes.
- Report whether formal evaluation is warranted now and list what must be clarified or fixed first.
Sources
- Qualitative Research & Evaluation Methods (Patton, 2002)
Also known as
- Exploratory evaluation
Categories
- Research Operations
- Research Design & Rigor