Bayesian model evidence as a practical alternative to deviance information criterion

Publisher
Royal Society Publshing
Abstract
While model evidence is considered by Bayesian statisticians as a gold standard for model selection (the ratio in model evidence between two models giving the Bayes factor), its calculation is often viewed as too computationally demanding for many applications. By contrast, the widely used deviance information criterion (DIC), a different measure that balances model accuracy against complexity, is commonly considered a much faster alternative. However, recent advances in computational tools for efficient multi-temperature Markov chain Monte Carlo algorithms, such as steppingstone sampling (SS) and thermodynamic integration schemes, enable efficient calculation of the Bayesian model evidence. This paper compares both the capability (i.e. ability to select the true model) and speed (i.e. CPU time to achieve a given accuracy) of DIC with model evidence calculated using SS. Three important model classes are considered: linear regression models,mixed models and compartmentalmodels widely used in epidemiology. While DIC was found to correctly identify the true model when applied to linear regression models, it led to incorrect model choice in the other two cases. On the other hand, model evidence led to correct model choice in all cases considered. Importantly, and perhaps surprisingly, DIC andmodel evidence were found to run at similar computational speeds, a result reinforced by analytically derived expressions.
Year
2018
Category
Refereed journal
Output Tags
Theme 2: Productive and Sustainable Land Management and Rural Economies (RESAS 2
WP 2.2 Livestock production, health, welfare and disease control (RESAS 2016-21)
RD 2.2.3 Disease mechanisms (RESAS 2016-21)
Audience: Scientific