Algorithmic Experience Research

Description

Longitudinal study of how people perceive, theorise about and are affected by recommendation feeds and automated decisions, combining log data with interviews.

Algorithmic experience research investigates the user side of algorithmic systems: whether people know an algorithm is shaping what they see, the folk theories they hold about it, how they try to steer it, and how its outputs affect behaviour, mood and wellbeing over time. Because effects accumulate slowly and are personalised, single-session usability tests miss them. Rigorous designs combine longitudinal diaries or experience sampling with logged exposure data, folk-theory elicitation interviews, and, where possible, controlled variation of the algorithm (for example a chronological feed holdout). Analysis links what people believe about the system to what it actually does, revealing misattribution, learned helplessness or gaming. The approach is well developed in social computing research but rarely operationalised in product UX teams.

Attributes

Loss type
Latent
Data
Mixed
Effort
Months

Steps

  1. Define the algorithmic surface (feed ranking, recommendations, pricing, moderation) and the experience outcomes of interest (control, trust, wellbeing, diversity).
  2. Recruit 15-30 participants and obtain consent to pair their exposure logs with self-reports.
  3. Run baseline interviews eliciting folk theories: what the participant thinks drives what they see, and how they try to influence it.
  4. Collect 2-6 weeks of diary or experience-sampling entries tied to specific algorithmic moments, alongside logged exposure.
  5. Where feasible, introduce a controlled variation (for example a holdout with reduced personalisation) and compare experience measures.
  6. Analyse gaps between folk theories and actual system behaviour, and link exposure patterns to experience outcomes.
  7. Translate findings into transparency, control and ranking-objective recommendations.

Sources

  • AI for Humans
  • The Alignment Problem
  • Eslami et al., 'I always assumed that I wasn't really that close to [her]' (CHI 2015)
  • Alvarado & Waern, Towards Algorithmic Experience (CHI 2018)

Also known as

  • Algorithmic experience (AX)
  • Algorithm awareness research
  • Feed experience research

Categories

  • Analytics & Behaviour
  • Computational & Machine Learning