UX evidence graph (LLM-extracted, researcher-reviewed knowledge graph)

Description

A local language model turns UX text into typed nodes and evidence-cited edges; researchers approve or reject the links, and questions are answered only from retrieved, reviewed evidence.

An evidence graph stores research material as typed entities (user groups, tasks, features, pain points, causes, emotions, outcomes, recommendations, claims) joined by typed relations such as causes, struggles with, supports or contradicts. In Rafiei's open-source prototype a language model running locally proposes these nodes and edges chunk by chunk, and every edge must carry the quoted evidence, its source document and chunk, a confidence and a weight. Nothing the model produces counts as a finding until a researcher reviews it: edges start as pending and are approved, edited or rejected through a spreadsheet round-trip, and duplicate nodes are merged only on explicit approval. Questions are answered by combining vector search over the text with graph search that favours approved edges and ignores rejected ones, and the model is told to answer from that context alone, cite sources and flag pending links. Its rigour comes from traceability and the human review gate, not from the extraction itself.

Attributes

Loss type
Latent
Effort
Days

Steps

  1. Collect the corpus (interview transcripts, open survey comments, support tickets, app reviews, usability notes), remove personal data, and initialise the project with a locally served chat model so no text leaves the machine.
  2. Ingest the files, splitting them into overlapping chunks of about 900 words, and run extraction so the model proposes typed nodes and typed edges, each edge with cited evidence text, source document and chunk.
  3. Export the edges to CSV and review every one against its quoted evidence: set it to approved, edited or rejected, correct the relation type, and adjust confidence (0 to 1) and weight before importing the file back.
  4. Generate merge suggestions for near-duplicate nodes, approve only merges that mean the same thing in context, and review nodes the same way as edges.
  5. Embed the chunks and ask research questions; check that each answer cites retrieved evidence, separates strong from weak support and flags any pending links it relied on.
  6. Export the graph (GraphML or JSON), node, edge and adjacency tables and the summary, and report only approved relations as findings, keeping pending ones as hypotheses to test.

Sources

  • ux-evidence-graph (Rafiei, GitHub repository, 2026)

Also known as

  • UX evidence graph
  • LLM-extracted UX knowledge graph
  • Researcher-reviewed evidence graph
  • Graph-grounded question answering over research text
  • ux-evidence-graph

Categories

  • User Research
  • Qualitative Analysis & Synthesis