Description
A blind test in which judges pick the odd one of three specimens; randomisation fixes chance success at one third, and repeated trials are tested against the binomial.
The triangle test answers a question that precedes any preference test: can people perceive a difference at all? Each trial presents three specimens, two of version A and one of B (or the reverse), with the odd specimen and the order chosen at random. The judge, blind to the allocation, names the odd one using a protocol fixed in advance. Because of the randomisation, the probability of a correct pick under no discrimination is exactly 1/3 whatever other differences exist between specimens. A single trial carries little evidence, so the design is repeated across many trials or judges and the number correct is referred to the binomial with p = 1/3. Repetition, not a larger set within one trial, is how sensitivity is gained. Its rigour rests entirely on the randomisation being carried out properly.
Attributes
- Discipline
- Experimentation & Causal Inference
- Loss type
- Siloed
- Stage
- Design, Evaluation
- Data
- Quantitative
- Effort
- Days
Steps
- Prepare three specimens per trial: two of version A and one of version B, balancing across trials which version is doubled.
- Randomise which specimen is the odd one and the order of presentation for every trial.
- Keep the judge blind and have them name the odd one using a response protocol fixed in advance (forced choice, no 'cannot tell').
- Score each trial correct or incorrect.
- Repeat over many trials or judges; as a rough guide, around 30 or more judgements are needed to detect moderate discriminability.
- Refer the number correct to the binomial distribution with p = 1/3 to obtain a one-sided p-value.
Sources
- Design and Analysis of Experiments, Vol. 1 (Hinkelmann & Kempthorne, 2008)
Also known as
- Triangle test
- Triangular test
- Odd-one-out test
- Triangle discrimination test
Categories
- Analytics & Behaviour
- Experimentation & Causal Inference