Jupyter notebooks vs. scripts (analysis medium choice)

Description

A deliberate decision about when analysis lives in exploratory notebooks and when it is promoted to tested, importable scripts, so UX analyses stay both explorable and reproducible.

Notebooks interleave code, output and narrative, which suits exploration and communicating analysis to stakeholders. Their weaknesses are hidden state (cells run out of order), poor diffing in version control and code duplicated across notebooks. Scripts and modules are linear, testable and diffable but less explorable. The rigorous practice is a two-tier workflow: notebooks for exploration and reporting, numbered and dated, and reusable logic (cleaning, scoring, modelling functions) promoted into a package that notebooks import. Before any result is reported, the notebook is restarted and run top to bottom, or executed headlessly, to prove it reproduces. Studies of public notebooks (Pimentel et al., 2019) found only a small minority reproduced the original results, which is the empirical case for these rules.

Attributes

Loss type
Unmigrated
Effort
Hours

Steps

  1. Use notebooks for exploration and name them with a sequence number, date and initials (e.g. 01-2026-03-04-vg-sus-explore.ipynb).
  2. Move any function used in more than one notebook, or longer than about 20 lines, into a src/ package with tests.
  3. Clear outputs or use a tool such as jupytext or nbstripout before committing so diffs stay readable.
  4. Restart the kernel and run all cells before sharing any figure or number; treat a failure as a bug.
  5. Execute report notebooks headlessly (e.g. with papermill or nbconvert) in the pipeline so results are regenerated, not hand-edited.
  6. Use scripts or pipeline tasks for production jobs such as nightly log aggregation or batch transcription.

Sources

  • Python (pandas, numpy, pathlib)
  • Jupyter Lab
  • Git
  • conda/venv
  • cookiecutter project templates
  • Wilson, G. et al. (2017) Good enough practices in scientific computing. PLOS Computational Biology
  • Pimentel, J. F. et al. (2019) A large-scale study about quality and reproducibility of Jupyter notebooks. MSR 2019
  • The Inquiry Series, C1 — The Python Research Environment — Setup, Philosophy & Reproducibility

Also known as

  • Notebook vs. script workflow
  • Literate computing vs. modular code

Categories

  • Research Operations
  • Research Design & Rigor