Psychometric Meta-Analysis (Hunter-Schmidt Artifact Correction)

Description

Correct pooled correlations for measurement error, range restriction and other artifacts to estimate relations between constructs rather than between imperfect measures.

Observed correlations understate true relations because of study artifacts. Unsystematic artifacts such as sampling error and data errors average out; at least ten systematic artifacts, notably measurement unreliability and range restriction, attenuate correlations by a multiplier. Psychometric meta-analysis corrects each study individually when artifact values are available, or, when they are not, meta-analyses observed correlations and then corrects the mean and variance using distributions of artifact values. Variance due to sampling error and artifact variation is removed, leaving the residual SD of true correlations, which shows whether a relation generalises. The approach originated in validity generalisation for personnel tests. Without corrections, meta-analytic inputs to path models are biased. The correct type of reliability coefficient must be used: inappropriate coefficients undercorrect.

Attributes

Loss type
Unmigrated
Stage
Strategy
Effort
Weeks

Steps

  1. Gather artifact values (reliabilities of both measures, range-restriction ratios) from the studies or other sources.
  2. If values are available for nearly all studies, correct each correlation and its sampling variance and meta-analyse the corrected values.
  3. Otherwise meta-analyse observed correlations and divide the mean by the mean compound attenuation multiplier.
  4. Correct the variance by removing sampling-error variance and variance due to artifact variation.
  5. Report the corrected mean correlation and the residual SD of true correlations with credibility intervals.

Sources

  • The Handbook of Research Synthesis and Meta-Analysis (Cooper et al., 2009)

Also known as

  • Correcting for study artifacts
  • Artifact distribution meta-analysis
  • Validity generalisation
  • Correction for attenuation in meta-analysis

Categories

  • Analytics & Behaviour
  • Statistics & Modelling