Description
Runs anonymous, multi-round expert questionnaires with statistical feedback between rounds to converge on consensus or map the structure of expert disagreement.
The Delphi Method, developed at the RAND Corporation for technological forecasting, structures expert judgement to avoid the distortions of face-to-face groups: dominance, conformity and halo effects. A panel of typically 10 to 25 experts answers an open first-round questionnaire. Responses are synthesised into rated items for round two, and after each round participants receive the group's median and interquartile range alongside their own previous rating, then re-rate and justify outlying positions. Anonymity is maintained throughout. Iteration stops at a pre-defined consensus criterion, such as an IQR of 2 or less on a 9-point scale, or when ratings stabilise. Items that never converge are reported as genuine disagreement. Rigour comes from pre-registered consensus thresholds, careful panel selection and controlled feedback.
Attributes
- Discipline
- Participatory, Design & Futures
- Loss type
- Siloed
- Stage
- Strategy, Definition
- Data
- Mixed
- Effort
- Weeks
Steps
- Round 1 — Open elicitation: Open-ended questionnaire to 10–25 experts. Ask them to generate criteria, factors, or estimates. Analyse responses using thematic analysis.
- Prepare Round 2 instrument: Compile Round 1 responses into a structured questionnaire. Participants rate or rank each item.
- Round 2 — Structured rating: Compute median, range, and IQR for each item. These statistics are the controlled feedback.
- Feedback and Round 3: Return statistics with participants' own previous rating highlighted. Ask them to reconsider in light of the group's distribution.
- Consensus criterion: IQR ≤ 2 on a 9-point scale. Items not reaching consensus are reported with the distribution of expert views — the disagreement structure is itself a finding.
Sources
- Preparing, Conducting, and Analyzing Delphi Surveys (PMC 2021)
- Applying the Delphi Method in Design Research (designorate.com)
- Preparing
- Conducting
- and Analyzing Delphi Surveys (PMC 2021)
- Veal, A. J. & Darcy, S., Research Methods in Sport Studies and Sport Management (2014)
Also known as
- Delphi technique
- Delphi survey
- Expert consensus panel
Categories
- Design
- Participatory, Design & Futures