Description
Tests whether categorical outcomes differ across groups, such as preference by device or success by design.
The chi-square test compares observed counts in a contingency table with the counts expected if the variables were unrelated. Fisher's exact test computes the exact probability for small tables. Bruce and Bruce present a resampling approach that shuffles labels to build the null distribution directly, which makes the logic intuitive. For 2×2 tables Sauro and Lewis recommend the N-1 chi-square over both the classic test and Fisher's test; for larger tables the standard chi-square applies when expected counts are adequate.
Attributes
- Discipline
- Statistics & Modelling
- Loss type
- Active
- Goal
- Compare designs, Segment users
- Stage
- Evaluation
- Data
- Quantitative
- Effort
- Hours
Steps
- Build the contingency table of counts.
- Check expected counts; for small 2×2 tables use the N-1 variant, for tiny tables use Fisher's exact test.
- Compute the statistic and p-value (or simulate by permutation).
- Inspect standardised residuals to see which cells drive the result.
- Report counts, percentages and an effect size such as Cramér's V.
Sources
- Practical Statistics for Data Scientists (Bruce & Bruce)
- Quantifying the User Experience (Sauro & Lewis)
- Companion Book to Quantifying the User Experience (Lewis & Sauro)
- Myatt, G., Making Sense of Data: A Practical Guide to Exploratory Data Analysis and Data Mining (2007)
- Gratton, C. & Jones, I., Research Methods for Sports Studies (2010)
- Veal, A. J. & Darcy, S., Research Methods in Sport Studies and Sport Management (2014)
Also known as
- Pearson chi-square test
- Fisher-Irwin test
- Test of independence
Categories
- Analytics & Behaviour
- Statistics & Modelling