Log File Analysis

Description

Reconstructs user behaviour, errors and usage patterns directly from raw server, application or search logs, independent of analytics tool sampling and schemas.

Before packaged analytics, researchers studied behaviour from raw logs: web server requests, application events and search queries. Transaction log analysis in library catalogues and information retrieval pioneered rigorous techniques such as session reconstruction, query reformulation analysis and failure rate measurement. Today logs remain the richest source where analytics tools are absent or inadequate: APIs, developer tools, back-end errors, search systems and legacy software. The method involves parsing logs, reconstructing sessions with sensible timeout rules, classifying events and computing behavioural measures. Rigour comes from documenting cleaning decisions (bot filtering, deduplication, session timeout, typically 30 minutes), validating reconstructed sessions against known cases, protecting privacy, and acknowledging that logs show actions but not intentions.

Attributes

Loss type
Siloed
Stage
Post-launch
Effort
Days

Steps

  1. Identify available logs and their fields, and agree privacy and access controls.
  2. Parse and clean logs: remove bots and health checks, normalise identifiers and timestamps.
  3. Reconstruct sessions using a documented timeout rule (often 30 minutes of inactivity).
  4. Classify events and compute measures such as error rates, query reformulations, task completion proxies and usage frequency.
  5. Validate reconstructions against a sample of known sessions or test accounts.
  6. Report patterns and follow up with qualitative research to explain them.

Sources

  • Practical Web Analytics for User Experience
  • Search log analysis: What it is, what's been done, how to do it (Bernard Jansen)

Also known as

  • Log Analysis
  • Server log analysis
  • Transaction log analysis

Categories

  • Analytics & Behaviour
  • Product Analytics & Strategy