Operationalization protocol

Description

Translates an abstract construct into specific, observable indicators and procedures, documenting how each indicator maps to the construct and what it leaves out.

Operationalisation is the bridge between a concept such as 'ease of use' and data. A rigorous protocol specifies the construct's dimensions, chooses indicators for each, defines the exact measurement procedure (instrument, wording, timing, scoring, units), and states the expected relationships between indicators. Lazarsfeld's sequence of imagery, specification of dimensions, selection of indicators, and formation of indices remains a sound template. Good protocols also record what each indicator misses (construct under-representation) and what irrelevant variance it may capture. In UX the step is commonly skipped: teams pick a familiar metric because it is available, leading to measures that do not represent what stakeholders mean.

Attributes

Loss type
Vestigial
Effort
Days

Steps

  1. Write a one-paragraph conceptual definition of the construct, citing literature or stakeholder interviews.
  2. Break it into dimensions (for example ease of use as efficiency, learnability, error tolerance and perceived effort).
  3. Select at least two indicators per dimension, mixing behavioural and attitudinal where possible.
  4. Specify the exact procedure for each indicator: instrument, wording, scale, timing, scoring rule and unit.
  5. State expected correlations between indicators (convergent) and with unrelated measures (discriminant), and pilot with 5-10 participants.
  6. Record known gaps and threats, and freeze the protocol before main data collection.

Sources

  • Dix, A., Statistics for HCI: Making Sense of Quantitative Data (2020)
  • Sauro, J. & Lewis, J. R., Quantifying the User Experience (2nd ed., 2016)
  • Leavy, P., Research Design: Quantitative, Qualitative, Mixed Methods, Arts-Based, and Community-Based Participatory Research Approaches (2017)
  • Statistics for HCI (Alan Dix)
  • Quantifying the User Experience (Sauro & Lewis)
  • Research Design (Leavy)
  • The Inquiry Series, Ch 8 — Measurement Theory — Before You Measure Anything
  • Yin, R., Qualitative Research from Start to Finish, 2nd ed. (2016)
  • Gratton, C. & Jones, I., Research Methods for Sports Studies (2010)
  • Veal, A. J. & Darcy, S., Research Methods in Sport Studies and Sport Management (2014)

Also known as

  • Operationalisation
  • Construct operationalisation
  • Measurement specification

Categories

  • Research Operations
  • Research Design & Rigor