Optimal Design Criteria and Design Efficiency (A-, D-, E-Optimality)

Description

Compare candidate experimental layouts before running them by how precisely they estimate the comparisons that matter, using the efficiency factor and A-, D- or E-criteria.

Optimal design theory scores a design's information matrix before any data are collected. The efficiency factor of a block design is the average variance of pairwise treatment differences under an unblocked design with the same replication divided by that under the candidate; it has a known upper bound, so a design's closeness to the best possible can be judged. D-optimality maximises the determinant of the information matrix (smallest joint confidence region), A-optimality minimises the average variance of the contrasts, and E-optimality minimises the variance of the worst-estimated contrast. Balanced designs such as BIBDs, Latin squares and Youden squares are optimal on all three (Kiefer's universal optimality). For test-versus-control designs, A-optimality is applied to control comparisons only. Hinkelmann and Kempthorne warn that optimality is tied to one criterion while real experimenters have several, so a near-balanced design with checked efficiency is often the practical answer.

Attributes

Loss type
Siloed
Stage
Strategy, Design
Effort
Days

Steps

  1. State the comparisons of interest: all pairs, test versus control, or model parameters.
  2. List candidate designs that respect practical constraints such as block size, number of participants or sessions.
  3. Compute each candidate's information matrix from its incidence or model matrix.
  4. Score candidates on the efficiency factor against its upper bound and on the A-, D- or E-criterion that matches the inferential goal.
  5. Prefer a balanced design where one exists; otherwise choose a candidate with efficiency close to the bound (above about 0.8 is a common target), weighing other practical criteria.
  6. Record the chosen design's efficiency in the study plan so loss from dropouts can be assessed later.

Sources

  • Design and Analysis of Experiments, Vol. 2 (Hinkelmann & Kempthorne, 2005)
  • Nonlinear Regression Analysis and Its Applications (Bates & Watts, 1988)
  • Nonlinear Regression (Seber & Wild, 1989)
  • Robust Statistics (Huber, 1981)
  • Advanced Calculus with Applications in Statistics (Khuri, 2003)

Also known as

  • Design efficiency factor
  • Optimal design criteria
  • D-optimality
  • A-optimality
  • E-optimality
  • Universal (Kiefer) optimality

Categories

  • Research Operations
  • Research Design & Rigor