Description
Compare the data's support for a theory-scaled effect against no effect, so a null result can count as evidence of absence, and report how robust that conclusion is.
A non-significant result alone cannot say whether an effect is absent or the study was insensitive. Dienes models the alternative hypothesis from theory: for a directional prediction, a half-normal distribution with mode zero and standard deviation equal to the expected effect, or half the plausible maximum. The maximum can come from room on the scale (a control mean of 4 on a 1 to 7 scale leaves at most 3, so SD 1.5) or from an original study when testing a replication. The Bayes factor is computed from the mean difference and its standard error. B above 3 is moderate evidence for the effect, below 1/3 moderate evidence for none, and between is insensitive. A robustness region reports the range of scales giving the same conclusion. Bayes factors are unaffected by stopping rules, so data can be collected until B is decisive.
Attributes
- Discipline
- Statistics & Modelling
- Loss type
- Siloed
- Stage
- Evaluation, Post-launch
- Data
- Quantitative
- Effort
- Hours
Steps
- Decide the direction and plausible scale of the effect from theory, the scale's room to move, or a prior study.
- Compute the mean difference and its standard error from the data.
- Calculate B with a half-normal model of H1 (for example with an online calculator), written B_HN(0, SD).
- Interpret: B > 3 evidence for an effect, B < 1/3 evidence for no effect, otherwise insensitive.
- Report the robustness region of SDs that lead to the same conclusion.
- Optionally continue collecting until B passes 10 or 1/10, combining related tests or raising thresholds when testing many questions.
Sources
- Cambridge Handbook of Research Methods (Nichols & Edlund, 2023)
- Structural Equation Modeling: A Bayesian Approach (Lee, 2007)
- Applied Bayesian Modelling (Congdon, 2003)
- Modes of Parametric Statistical Inference (Geisser, 2006)
- Bayesian Analysis for the Social Sciences (Jackman, 2009)
- Bayesian Statistical Modelling (Congdon, 2006)
Also known as
- Bayes Factor with a Theory-Scaled Half-Normal Prior and Robustness Region
- Dienes Bayes factor
- Evidence for the null
- B_HN
Categories
- Analytics & Behaviour
- Statistics & Modelling