Description
Compare candidate experimental layouts before running them by how precisely they estimate the comparisons that matter, using the efficiency factor and A-, D- or E-criteria.
Optimal design theory scores a design's information matrix before any data are collected. The efficiency factor of a block design is the average variance of pairwise treatment differences under an unblocked design with the same replication divided by that under the candidate; it has a known upper bound, so a design's closeness to the best possible can be judged. D-optimality maximises the determinant of the information matrix (smallest joint confidence region), A-optimality minimises the average variance of the contrasts, and E-optimality minimises the variance of the worst-estimated contrast. Balanced designs such as BIBDs, Latin squares and Youden squares are optimal on all three (Kiefer's universal optimality). For test-versus-control designs, A-optimality is applied to control comparisons only. Hinkelmann and Kempthorne warn that optimality is tied to one criterion while real experimenters have several, so a near-balanced design with checked efficiency is often the practical answer.
Attributes
- Discipline
- Research Design & Rigor
- Loss type
- Siloed
- Data
- Analytical / expert
- Effort
- Days
Steps
- State the comparisons of interest: all pairs, test versus control, or model parameters.
- List candidate designs that respect practical constraints such as block size, number of participants or sessions.
- Compute each candidate's information matrix from its incidence or model matrix.
- Score candidates on the efficiency factor against its upper bound and on the A-, D- or E-criterion that matches the inferential goal.
- Prefer a balanced design where one exists; otherwise choose a candidate with efficiency close to the bound (above about 0.8 is a common target), weighing other practical criteria.
- Record the chosen design's efficiency in the study plan so loss from dropouts can be assessed later.
Sources
- Design and Analysis of Experiments, Vol. 2 (Hinkelmann & Kempthorne, 2005)
- Nonlinear Regression Analysis and Its Applications (Bates & Watts, 1988)
- Nonlinear Regression (Seber & Wild, 1989)
- Robust Statistics (Huber, 1981)
- Advanced Calculus with Applications in Statistics (Khuri, 2003)
Also known as
- Design efficiency factor
- Optimal design criteria
- D-optimality
- A-optimality
- E-optimality
- Universal (Kiefer) optimality
Categories
- Research Operations
- Research Design & Rigor