Description
Infer a speaker's emotion or arousal and valence from recorded speech by computing prosodic, spectral and voice-quality features over short windows, summarising them per utterance and classifying them.
Emotion shows strongly in how things are said. Speech emotion recognition splits each utterance into short windows treated as stationary, computes low-level descriptors (pitch, intensity, mel-frequency cepstral coefficients, spectral centroid and flux, pauses, harmonics-to-noise ratio) and summarises them with statistics over the utterance, which are then classified into discrete emotions or levels of arousal and valence. Pitch and energy features matter most. Speaker-dependent models can be very accurate, but practical speaker-independent accuracy is modest; the book cites about 81% on an acted German database, while human listeners manage about 60% on such tasks. High-arousal states such as anger are easier to detect than joy or fear, and global statistics lose timing detail. For UX the method adds a channel to self-report and transcript analysis when speech is already recorded.
Attributes
- Discipline
- Neuro, Biometrics & Affect
- Loss type
- Siloed
- Stage
- Evaluation, Post-launch
- Data
- Quantitative
- Effort
- Weeks
Steps
- Record speech with consent and segment it into utterances; note recording conditions.
- Normalise the voice signal per speaker and threshold intensity to keep voiced samples.
- Compute low-level descriptors over short overlapping windows: pitch, intensity, MFCCs, spectral centroid and flux, pause durations and harmonics-to-noise ratio.
- Summarise descriptors per utterance with statistics (means, SDs, quartiles, slopes).
- Classify into emotions or arousal and valence levels with an SVM, GMM or recurrent network trained speaker-independently.
- Validate with cross-validation and confusion matrices, and compare with human ratings on a sample.
Sources
- Multimodal Sentiment Analysis (Poria et al., 2018)
Also known as
- Vocal affect analysis
- Speech emotion recognition
- Paralinguistic emotion detection
- Acoustic-prosodic affect recognition
Categories
- Analytics & Behaviour
- Neuro, Biometrics & Affect