Description
Improve a running process by repeatedly running small factorial changes around current settings and moving the operating point towards better conditions.
EVOP is a hybrid of monitoring and experimentation: small designed experiments run on the live process, by its own operators, as part of normal work. A two-level factorial in two or three factors is run around a centre point at current settings, with changes small enough not to harm output. Because the signal is weak against process noise, cycles are repeated until effects stand out; the operating point then moves to the better conditions and a new design is centred there, continuing until no further gain appears. Its design properties are attractive, but the small region and few factors risk confounded effects and missed interactions, and it demands a long, ideally permanent programme with management commitment. In spirit it anticipates continuous online experimentation and hill-climbing on product parameters.
Attributes
- Discipline
- Experimentation & Causal Inference
- Loss type
- Unmigrated
- Stage
- Post-launch
- Data
- Quantitative
- Effort
- Months
Steps
- Choose two or three parameters that can be varied slightly without harming the user experience or business results.
- Run a two-level factorial with a centre point at current settings as part of normal operation.
- Repeat cycles, accumulating data until effects exceed their error limits.
- Move the operating point to the better conditions and centre a new design there.
- Continue until no further improvement is found, and log each move so the path can be audited.
- Review factors periodically so interactions outside the small region are not missed.
Sources
- Statistical Design and Analysis of Experiments (Mason et al., 2003)
Also known as
- EVOP
Categories
- Analytics & Behaviour
- Experimentation & Causal Inference