MUSHRA Listening Test (ITU-R BS.1534)

Description

A standardised listening test in which trained listeners rate several audio versions against a known reference, with a hidden reference and low-quality anchors to validate ratings.

MUSHRA asks listeners to rate multiple stimuli on a 0-100 continuous quality scale, simultaneously, with free switching between them and an open reference. Among the stimuli are a hidden copy of the reference and one or more anchors (e.g. a 3.5 kHz low-pass filtered version). Listeners who fail to rate the hidden reference near 100 are excluded under post-screening rules. The design yields sensitive, reliable comparisons for intermediate audio quality. For UX, it is a model of rigour for any perceptual comparison: voice assistant TTS, notification sounds, call quality, audio in video products. Its anchors and hidden reference make rater reliability measurable.

Attributes

Loss type
Unmigrated
Stage
Evaluation
Effort
Days

Steps

  1. Prepare stimuli: the reference, the systems under test, a hidden reference and the standard anchors, loudness-normalised.
  2. Recruit and train listeners (typically 15-20 experienced listeners) with a familiarisation phase.
  3. Run trials with all stimuli for an item presented together in random order, rated on 0-100.
  4. Apply post-screening: exclude listeners rating the hidden reference below 90 for more than 15% of items.
  5. Analyse with mixed models or ANOVA, reporting means and 95% confidence intervals per system.
  6. Relate quality differences to perceptual attributes through follow-up descriptive analysis if needed.

Sources

  • ITU-R BS.1534-3 (2015) Method for the subjective assessment of intermediate quality level of audio systems
  • Bech & Zacharov, Perceptual Audio Evaluation: Theory, Method and Application (2006)

Also known as

  • MUSHRA
  • Multiple Stimuli with Hidden Reference and Anchor
  • ITU-R BS.1534

Categories

  • Analytics & Behaviour
  • Neuro, Biometrics & Affect