Data Mining Project Process (Define, Prepare, Implement, Deploy)

Description

A four-stage plan for analytics projects: define the problem and deliverables, prepare the data, select and optimise analyses, then deploy and measure the results.

Myatt frames every exploratory analysis or data mining project as four linked steps, with iteration between them. Problem definition breaks a headline goal into questions the data can answer, sets specific measurable success criteria tied to a business objective, states deliverables (report, rules, embedded model, new workflow), the accuracy needed and the relative cost of the two kinds of wrong prediction, checks the data represent the target population, and assigns roles: project leader, subject matter expert, analyst, IT expert and consumer of the results. A plan adds timetable, constraints, privacy issues, risks and a cost-benefit case. Data preparation characterises, cleans, transforms and segments the data. Implementation selects among summary tables, graphs, statistics, grouping and prediction methods according to whether the task is summarising, finding hidden relationships or predicting, and optimises them. Deployment delivers a report or integrated model, plans change management and training, measures business impact, ideally with a randomised field comparison of users with and without the model, monitors models as data age, and ends with a project review. For UX analytics it gives a scoping discipline that ties exploratory work to decisions.

Attributes

Loss type
Siloed
Data
Mixed
Effort
Weeks

Steps

  1. Write the headline objective and break it into answerable data questions with measurable success criteria.
  2. Define deliverables, required accuracy, speed constraints and the consequences of each type of error.
  3. Assemble a cross-disciplinary team, agree roles and share a glossary of terms.
  4. Plan: data sources, privacy and legal issues, timetable with review points, risks and cost-benefit.
  5. Prepare the data, then select, run and optimise the analysis methods that fit the questions.
  6. Deploy results as a report, rules or integrated model, with training and change management.
  7. Measure impact against the success criteria (ideally with a randomised comparison), monitor over time and run a project review.

Sources

  • Myatt, G., Making Sense of Data: A Practical Guide to Exploratory Data Analysis and Data Mining (2007)

Also known as

  • Myatt's four-step data analysis process
  • Exploratory data analysis project lifecycle
  • Data mining project plan
  • CRISP-DM-style process

Categories

  • Research Operations
  • Research Design & Rigor