Replication Study (Direct and Conceptual)

Description

Deliberately repeat an earlier study, with the same methods (direct) or different methods testing the same claim (conceptual), to confirm or disconfirm its result.

Replication is the purposeful repetition of earlier research to corroborate or disconfirm it. Direct replication repeats the methods as closely as possible; conceptual replication tests the same hypothesis with other methods, samples or measures. Many findings do not hold up, yet replications are rare (about 1 per cent of psychology articles and 0.13 per cent in education) because of publication and career incentives and poor data sharing. Choose studies by importance, decision relevance and the availability of protocols, instruments and data. Build some direct replication into new studies, and share your own protocols and data. Replicating with an under-researched population is a recognised contribution. In UX, many design 'laws' and internal benchmark findings rest on single studies.

Attributes

Loss type
Siloed
Data
Mixed
Effort
Weeks

Steps

  1. Select a study by importance, how much decisions depend on it, and whether protocol, instruments and data are available.
  2. Decide on direct replication (same methods) or conceptual replication (same claim, different methods or population).
  3. Obtain or rebuild the protocol and instruments, and preregister the comparison if possible.
  4. Run the study and compare effect sizes and conclusions with the original, not just significance.
  5. Publish or archive protocols and data so others can replicate your work.

Sources

  • Research Design (Leavy, 2017)
  • MIT Psychology (Stangor, 2012)

Also known as

  • Direct replication
  • Conceptual replication
  • Replication research

Categories

  • Research Operations
  • Research Design & Rigor