Randomised Triangle (Odd-One-Out) Discrimination Test

Description

A blind test in which judges pick the odd one of three specimens; randomisation fixes chance success at one third, and repeated trials are tested against the binomial.

The triangle test answers a question that precedes any preference test: can people perceive a difference at all? Each trial presents three specimens, two of version A and one of B (or the reverse), with the odd specimen and the order chosen at random. The judge, blind to the allocation, names the odd one using a protocol fixed in advance. Because of the randomisation, the probability of a correct pick under no discrimination is exactly 1/3 whatever other differences exist between specimens. A single trial carries little evidence, so the design is repeated across many trials or judges and the number correct is referred to the binomial with p = 1/3. Repetition, not a larger set within one trial, is how sensitivity is gained. Its rigour rests entirely on the randomisation being carried out properly.

Attributes

Loss type
Siloed
Effort
Days

Steps

  1. Prepare three specimens per trial: two of version A and one of version B, balancing across trials which version is doubled.
  2. Randomise which specimen is the odd one and the order of presentation for every trial.
  3. Keep the judge blind and have them name the odd one using a response protocol fixed in advance (forced choice, no 'cannot tell').
  4. Score each trial correct or incorrect.
  5. Repeat over many trials or judges; as a rough guide, around 30 or more judgements are needed to detect moderate discriminability.
  6. Refer the number correct to the binomial distribution with p = 1/3 to obtain a one-sided p-value.

Sources

  • Design and Analysis of Experiments, Vol. 1 (Hinkelmann & Kempthorne, 2008)

Also known as

  • Triangle test
  • Triangular test
  • Odd-one-out test
  • Triangle discrimination test

Categories

  • Analytics & Behaviour
  • Experimentation & Causal Inference