Description
Choose the number of participants and prompts for a diary or ESM study by simulating many datasets from a multilevel model and counting how often the effect is detected.
Analytic power formulas exist only for simple multilevel questions. The simulation approach generates synthetic datasets from a hypothesised population model, fits the model to each, and takes power as the share of significant results. PowerAnalysisIL, an R Shiny app, covers 11 common ESM models with AR(1) level-1 errors, including group mean differences, random-slope effects, cross-level interactions and autoregressive effects; Bolger and Laurenceau do the same with Mplus MONTECARLO for five diary models, with at least 1,000 replications. Parameter values come from the literature, a pilot or a previous study fitted by REML. Outputs are power curves over participants and time points, with bias, standard errors and coverage. Two lessons recur: adding participants usually buys more power than adding days, and random slopes reduce power for the average slope. The exercise forces researchers to state every variance component, which is useful in itself.
Attributes
- Discipline
- Research Design & Rigor
- Loss type
- Unmigrated
- Goal
- Support a decision
- Data
- Analytical / expert
- Effort
- Days
Steps
- Specify the multilevel model for the hypothesis, including AR(1) errors.
- Get parameter values (fixed effects, variance components, autocorrelation) from literature, a pilot or prior data fitted with REML.
- Choose candidate numbers of participants and time points, using expected completed prompts, not scheduled ones.
- Run 1,000 or more Monte Carlo replicates per design, estimating run time first.
- Read power curves, bias and coverage for each design.
- Choose the design that reaches target power (typically 80%) within feasibility and burden limits.
Sources
- The Open Handbook of Experience Sampling Methodology (Myin-Germeys & Kuppens, 2022)
- Intensive Longitudinal Methods (Bolger & Laurenceau, 2013)
Also known as
- Simulation-based power analysis for ESM designs
- PowerAnalysisIL
- Monte Carlo power for multilevel intensive longitudinal designs
- Monte Carlo power analysis for intensive longitudinal designs
- Simulation-based power for diary studies
Categories
- Research Operations
- Research Design & Rigor