Publisher
Royal Society Publshing
Abstract
While model evidence is considered by Bayesian statisticians
as a gold standard for model selection (the ratio in model
evidence between two models giving the Bayes factor), its
calculation is often viewed as too computationally demanding
for many applications. By contrast, the widely used deviance
information criterion (DIC), a different measure that balances
model accuracy against complexity, is commonly considered
a much faster alternative. However, recent advances in
computational tools for efficient multi-temperature Markov
chain Monte Carlo algorithms, such as steppingstone sampling
(SS) and thermodynamic integration schemes, enable efficient
calculation of the Bayesian model evidence. This paper
compares both the capability (i.e. ability to select the true
model) and speed (i.e. CPU time to achieve a given accuracy)
of DIC with model evidence calculated using SS. Three
important model classes are considered: linear regression
models,mixed models and compartmentalmodels widely used
in epidemiology. While DIC was found to correctly identify
the true model when applied to linear regression models, it
led to incorrect model choice in the other two cases. On the
other hand, model evidence led to correct model choice in all
cases considered. Importantly, and perhaps surprisingly, DIC
andmodel evidence were found to run at similar computational
speeds, a result reinforced by analytically derived expressions.
Year
2018
Category
Refereed journal