Multi-pass analysis pipeline

Description

Splitting AI-assisted qualitative analysis into separate, checkable passes, such as segmenting, first-order coding, consolidation, second-order categorisation and synthesis, rather than one prompt.

Asking a model to 'find themes' in a corpus in one step produces unverifiable, often generic output. A multi-pass pipeline mirrors established qualitative cycles: pass one segments and cleans; pass two applies first-order codes segment by segment with evidence; pass three consolidates near-duplicate codes into a codebook; pass four groups codes into second-order categories; pass five drafts synthesis with links to evidence. Each pass has a narrow task, a schema, logged outputs and a human review point. Rigour comes from the separation (errors are traceable to a pass), from never letting later passes see only summaries without evidence, and from human decisions at the abstraction steps.

Attributes

Loss type
Latent
Stage
Definition
Effort
Weeks

Steps

  1. Define the passes and each pass's input, output schema and review owner.
  2. Pass 1: segment transcripts by speaker turn and remove moderator text and personal data.
  3. Pass 2: generate first-order codes per segment with verbatim evidence, validated against the schema.
  4. Pass 3: cluster and merge similar codes into a draft codebook; a researcher reviews merges.
  5. Pass 4: propose second-order categories with linked codes; researchers revise and approve.
  6. Pass 5: draft synthesis statements, each linked to supporting codes and quotes, and check counts of participants per theme.
  7. Log every pass's outputs and keep them so any theme can be traced back to segments.

Sources

  • Anthropic Claude API (claude-sonnet-4-20250514)
  • structured outputs (JSON schema)
  • asyncio
  • Saldaña, J. (2021) The Coding Manual for Qualitative Researchers, 4th ed.
  • Strauss, A. & Corbin, J. (1990) Basics of Qualitative Research
  • Gao, J. et al. (2024) CollabCoder: A lower-barrier, rigorous workflow for inductive collaborative qualitative analysis with large language models. CHI
  • The Inquiry Series, C12 — AI Agents for Structured Qualitative Analysis at Scale

Also known as

  • Staged LLM analysis
  • Iterative coding passes

Categories

  • User Research
  • Qualitative Analysis & Synthesis