Association Rule Mining

Description

Finds IF-THEN rules about which categorical values tend to occur together in records, and ranks them by support, confidence and lift.

Association rule mining is an unsupervised grouping method that groups records by combinations of categorical values and turns each group into IF-THEN rules, for example IF a user has used search AND saved a filter THEN they export a report. Every candidate rule is scored against the whole data set. Support is the share of all records covered by the full group. Confidence is the group support divided by the support of the IF-part, so it says how predictable the THEN-part is once the IF-part holds. Lift divides confidence by the support of the THEN-part alone, so it corrects for outcomes that are common anyway: values above 1 indicate a positive association and below 1 a negative one. A minimum group size keeps the search tractable but hides rules based on few records. Continuous variables must first be binned. For product teams the method turns feature-usage or event logs into readable statements about co-use, bundles and paths to an outcome, which are easier to discuss with stakeholders than cluster centroids. Rules describe co-occurrence, not cause, and the method can produce many rules that need prioritising and checking with domain knowledge or hypothesis tests.

Attributes

Loss type
Siloed
Effort
Days

Steps

  1. Build a table with one row per user, session or transaction and categorical columns (for example features used, plan, platform); bin any continuous variables.
  2. Set a minimum group size (support threshold) using what you learnt when characterising the data.
  3. Generate groups of value combinations that meet the threshold.
  4. Extract every possible IF-THEN rule from each group, putting one value in the THEN-part at a time.
  5. Calculate support, confidence and lift for each rule against the whole data set.
  6. Rank rules by confidence and lift, discard rules with lift near or below 1, and review the remainder with subject matter experts.
  7. Test promising rules with a hypothesis test or a controlled experiment before acting on them.

Sources

  • Myatt, G., Making Sense of Data: A Practical Guide to Exploratory Data Analysis and Data Mining (2007)

Also known as

  • Associative rules
  • Market basket analysis
  • Support-confidence-lift rule mining
  • Affinity analysis

Categories

  • Analytics & Behaviour
  • Computational & Machine Learning