Metacognitive Demand Profiling of GenAI Interactions

Description

Divide a GenAI workflow into stages, name the metacognitive ability each stage taxes, and measure users' confidence, calibration and reliance at each stage while varying interface properties.

Generative AI shifts work from producing to specifying, checking and deciding when to rely on output, which demands metacognition: awareness of one's goals, calibrated confidence in one's judgement and flexibility in strategy. This approach profiles those demands stage by stage: prompt formulation (self-awareness, task decomposition), prompt iteration (confidence and flexibility), output evaluation (confidence in one's own evaluation) and integration into one's workflow (awareness of when AI helps). For each, it pairs prospective measures such as predictions and judgements of learning with retrospective ones such as confidence ratings, debriefs and coded behaviour. Confidence is scored against objectively marked performance, giving calibration and resolution (meta-d'). Varying output length, generation speed, fluency or failure type then shows which interface properties inflate confidence or reliance without improving accuracy. The paper proposes this as a research agenda rather than a tested protocol.

Attributes

Loss type
Latent
Data
Mixed
Effort
Weeks

Steps

  1. Divide the target workflow into stages: prompt formulation, prompt iteration, output evaluation and reliance, and integration into the wider workflow.
  2. For each stage name the metacognitive ability it demands, such as self-awareness of goals, task decomposition, calibrated confidence or strategic flexibility.
  3. Choose prospective measures (predictions, judgements of learning, goal-setting worksheets) and retrospective measures (confidence ratings, debriefs, coded observation) for each ability.
  4. Collect confidence ratings alongside objectively scored performance and estimate calibration (confidence versus accuracy) and resolution (for example meta-d').
  5. Manipulate one interface factor at a time, such as output length, generation speed, fluency of wording or the type of model failure.
  6. Relate changes in confidence, evaluation accuracy and reliance to those factors, and report stages where users are over- or under-confident.

Sources

  • The Metacognitive Demands and Opportunities of Generative AI (Tankelevitch et al., 2024)

Also known as

  • Measuring metacognitive demands of generative AI
  • Confidence and reliance profiling for GenAI tools
  • Metacognitive demand mapping for AI workflows

Categories

  • User Research
  • Usability & Evaluation