Deviance Information Criterion (DIC) for Bayesian Model Comparison

Description

A fit-plus-complexity score for MCMC-fitted models: posterior mean deviance plus an estimated effective number of parameters, used to shortlist rival Bayesian models.

AIC and BIC need a parameter count, which is undefined in hierarchical models where random effects are partly pooled. DIC estimates complexity from the MCMC output itself: pD is the posterior mean deviance minus the deviance at the posterior means, so a set of heavily pooled random effects counts as far fewer than one parameter each. DIC is the deviance at the posterior means plus twice pD; lower is better. An alternative complexity estimate takes half the posterior variance of the deviance. Congdon uses DIC as a cheap first sift across many candidate models, then compares the few leading rivals more carefully by predictive criteria or marginal likelihoods. The value of pD is itself diagnostic: poorly identified parameters inflate it, and negative values warn that the criterion is unreliable for that model.

Attributes

Loss type
Siloed
Effort
Days

Steps

  1. Fit each candidate model by MCMC and confirm convergence before using any draws.
  2. Record the deviance (minus twice the log likelihood) at every retained iteration and average it.
  3. Evaluate the deviance at the posterior means of the parameters.
  4. Compute pD as mean deviance minus deviance at the means (or half the posterior variance of the deviance as a check), and DIC as mean deviance plus pD.
  5. Rank the models by DIC and inspect pD for implausible or negative values.
  6. Treat differences of only a few points as indecisive and compare the leading two or three models by posterior predictive checks, cross-validation or Bayes factors.

Sources

  • Applied Bayesian Modelling (Congdon, 2003)
  • Bayesian Statistical Modelling (Congdon, 2006)

Also known as

  • DIC
  • Deviance information criterion
  • Spiegelhalter DIC
  • pD (effective number of parameters)
  • Effective number of parameters

Categories

  • Analytics & Behaviour
  • Statistics & Modelling