Description
Converts rating responses to the share choosing the top one or two options, then analyses that share as a proportion.
Dashboards often report "percent who agree" (top-two-box on a 5-point scale) rather than means. To test a claim such as "at least 75% agree", convert 4s and 5s to 1 and everything else to 0, then use the binomial methods for completion rates: a mid-p test against the benchmark and an adjusted-Wald interval. Sauro and Lewis show the trade-off clearly: converting discards intensity, so the same data that gives a 72.5% chance of meeting the goal as top-box gives 81.4% when analysed as a mean with a t-test. Top-box is easy to communicate but needs larger samples to detect changes, so analyse raw means where you can and report top-box where dashboards require it.
Attributes
- Discipline
- Statistics & Modelling
- Loss type
- Active
- Stage
- Evaluation, Post-launch
- Data
- Quantitative
- Effort
- Hours
Steps
- Decide the top-box rule before analysis (for example 4 or 5 on a 5-point scale).
- Convert each response to 1 or 0.
- Compute the proportion with an adjusted-Wald interval.
- Test against any benchmark with the mid-p binomial test for small samples.
- Also report the mean and its interval so intensity is not lost.
Sources
- Quantifying the User Experience (Sauro & Lewis)
Also known as
- Top-box
- Top-two-box
- Net top-minus-bottom box
Categories
- Analytics & Behaviour
- Statistics & Modelling