Description
Before launch, 10 to 20 people read a fixed script of utterances to a voice system to tune timeouts and catch common misrecognitions.
A pre-launch check from interactive voice response practice that base recognition parameters are right before real users are exposed. A small set of speakers, typically 10 to 20, read a predetermined script covering the key utterance types: digit strings, names, domain terms and expected commands. Results are compared with the script to tune end-of-speech timeouts (phone numbers need longer pauses between digit groups) and to identify frequent misrecognitions, which are added to expected phrases or grammars. Even with cloud recognisers it catches domain-specific failures; Pearl's medical app heard 'vial' as 'vile' or 'file'. It complements rather than replaces pilot data, because scripted speech differs from spontaneous speech.
Attributes
- Discipline
- Usability & Evaluation
- Loss type
- Siloed
- Stage
- Evaluation
- Data
- Quantitative
- Effort
- Days
Steps
- Write a script covering key utterance types: digits, names, domain terms, commands and likely confusables.
- Recruit 10 to 20 speakers varied in accent, age and gender, matching the target population.
- Have each speaker read the script to the system in realistic acoustic conditions.
- Compare recognition results with the script and tally misrecognitions by item.
- Adjust end-of-speech timeouts and add misrecognised variants to expected phrases or grammars.
- Re-run problem items, then proceed to pilot testing with spontaneous speech.
Sources
- Designing Voice User Interfaces (Pearl, 2017)
Also known as
- Recognition testing with scripted speakers
- Scripted recognition test
Categories
- User Research
- Usability & Evaluation