Description
Write a question by moving from concept to assertion to request for an answer, then predict its reliability, validity and quality in the SQP program and revise.
A formal route from construct to question. Concepts are first classed as by postulation (needing several indicators) or by intuition (askable directly). Each concept by intuition is matched to a basic concept type (evaluation, feeling, behaviour, norm and so on), written as an assertion using the linguistic structure for that type, then turned into a balanced request for an answer. Remaining choices (introduction, instruction, scale type, number of categories, labels, don't know option) are made explicitly. The question's characteristics are then coded into SQP, a free predictor built on meta-analysis of thousands of multitrait-multimethod experiments, which estimates reliability, validity and overall quality and suggests changes. Versions are compared, including across translations. Predicted quality is well below 1, which also supplies correction factors for measurement error at analysis.
Attributes
- Discipline
- Survey & Psychometrics
- Loss type
- Unmigrated
- Stage
- Design
- Data
- Analytical / expert
- Effort
- Days
Steps
- Decide whether each concept is by postulation (several indicators needed) or by intuition (directly askable).
- Classify each concept by intuition as a basic concept and write an assertion using the matching linguistic structure.
- Turn the assertion into a balanced request for an answer.
- Decide the introduction, instructions, scale type, number of categories, labels and don't know option.
- Code the question's characteristics in SQP and read predicted reliability, validity and quality.
- Apply suggested changes, re-code and compare versions; for translated surveys compare country versions as well.
- Keep the predicted quality estimates to correct correlations for measurement error at analysis.
Sources
- The SAGE Handbook of Survey Methodology (Wolf et al., 2016)
Also known as
- Saris-Gallhofer question design procedure
- Survey Quality Predictor
- SQP
- Concept-by-intuition question writing
Categories
- Analytics & Behaviour
- Survey & Psychometrics