Description
Writing bespoke processing steps for an NLP pipeline, such as speaker filters, codebook taggers or anonymisers, so research-specific logic runs consistently over every transcript.
spaCy lets researchers register custom components that run inside the pipeline and add extension attributes to documents, sentences and tokens (e.g. doc._.participant_id, sent._.speaker, sent._.code_candidates). This turns study-specific rules into reusable, testable code: removing moderator turns, tagging sentences that match codebook definitions, redacting personal data, or attaching interview metadata. Rigour comes from unit tests for each component, clear documentation of rules, versioning components with the pipeline, and evaluating outputs on annotated samples just as for statistical components. Keeping such logic inside the pipeline, rather than in scattered post-processing scripts, also ensures every transcript receives identical treatment.
Attributes
- Discipline
- Computational & Machine Learning
- Loss type
- Unmigrated
- Goal
- Synthesise data
- Stage
- Definition
- Data
- Analytical / expert
- Effort
- Days
Steps
- Specify the component's job, inputs and outputs (e.g. add sent._.speaker from diarisation data).
- Register extension attributes and implement the component with @Language.component or a factory.
- Insert it at the correct pipeline position (e.g. after the parser and before downstream taggers).
- Write unit tests on small example texts covering normal and edge cases.
- Evaluate on a sample of real transcripts and record precision and recall where applicable.
- Package and version components with the pipeline configuration so others can reproduce runs.
Sources
- spaCy (en_core_web_trf transformer model)
- spacy-transformers
- textacy
- displacy visualisation
- spaCy documentation: Processing pipelines and custom components (spacy.io)
- The Inquiry Series, C10 — spaCy — Building NLP Pipelines for Qualitative Research
Also known as
- Custom pipeline components
- Extension attributes
Categories
- Analytics & Behaviour
- Computational & Machine Learning