Instrument Specification Frameworks for Mixed Methods

Description

Plan a questionnaire and its qualitative instruments with matched specification tables that fix constructs, items, analyses and cross-strand alignment before any item is written.

A planning discipline borrowed from test construction. The questionnaire specification table maps each section to its construct, definition, number and type of items, coding and planned analysis, with an extra column stating how it aligns with qualitative data. A matching qualitative table maps each area of interest to its definition, sources (interviews, focus groups, documents, observations), analysis strategy and alignment with the quantitative strand. Writing these tables first exposes constructs with no items, items with no analysis, and qualitative areas that can never be compared with the survey. Inferences are later legitimated through quantitative validation (for example Rasch or IRT), qualitative credibility checks and crossover analyses that test whether the two strands measure the same thing. The cost is effort before any data collection.

Attributes

Loss type
Siloed
Data
Mixed
Effort
Days

Steps

  1. Define each construct or area of interest and link it to a research question.
  2. Fill the questionnaire specification table: construct, definition, number and type of items, coding, planned analysis and alignment with qualitative data.
  3. Fill the matching qualitative specification table: area, definition, sources, analysis strategy and alignment with the quantitative strand.
  4. Check both tables for gaps: constructs without items, items without analysis, areas without a counterpart.
  5. Write, pilot and revise the instruments against the tables.
  6. Validate with quantitative, qualitative and crossover analyses and report which legitimation types were achieved.

Sources

  • The Sage Handbook of Mixed Methods Research Design (Poth, 2023)

Also known as

  • Instrument blueprint
  • Questionnaire specification framework
  • Instrument specification frameworks for mixed methods instrument development

Categories

  • Analytics & Behaviour
  • Survey & Psychometrics