Validated Scale Construction

Description

Develops multi-item measures through construct definition, item generation, expert review, piloting, factor analysis and reliability and validity testing.

The treatise's curriculum calls for validated scale construction and its borrowing matrix lists validated scales among clinical research methods. A rigorous process defines the construct and its boundaries, generates a large item pool, gathers expert content-validity ratings, runs cognitive interviews, pilots with a sizeable sample, conducts exploratory then confirmatory factor analysis on separate samples, tests reliability (internal consistency, test-retest) and establishes convergent, discriminant and criterion validity. UX often writes bespoke questions or borrows SUS, UMUX or others without checking fit for new contexts, which produces scores that are not comparable or not measuring what is claimed. Validated construction is the antidote.

Attributes

Loss type
Vestigial
Stage
Definition
Effort
Months

Steps

  1. Define the construct, its dimensions and what it excludes, based on theory and qualitative research.
  2. Generate an item pool several times larger than the final scale.
  3. Collect expert content-validity ratings and run cognitive interviews with target users.
  4. Pilot with 200+ respondents and run exploratory factor analysis; refine items.
  5. Confirm structure with CFA or IRT on a new sample; test reliability and invariance.
  6. Establish validity against related measures and behavioural criteria, then document norms.

Sources

  • Lost Methods in UX — A Complete Treatise (Gopinath)
  • DeVellis RF (2016) Scale Development: Theory and Applications. 4th ed
  • Cronbach LJ & Meehl PE (1955) Construct validity in psychological tests. Psych Bulletin
  • Boateng GO et al. (2018) Best practices for developing and validating scales. Frontiers in Public Health

Also known as

  • Psychometric scale development
  • Questionnaire validation

Categories

  • Analytics & Behaviour
  • Survey & Psychometrics