Adrian Raftery: Bayesian Model Averaging Research

Adrian Raftery: Research on Bayesian Model Averaging, Hypothesis Testing and Model Selection

I am interested in how Bayes factors may be used as an alternative to P-values and standard frequentist significance testing for testing hypotheses. It seems that P values can often be poorly calibrated in the long run, averaged over situations where effects both are and are not present, and that Bayes factors can provide a better calibrated alternative (Raftery and Zheng 2003; Viallefont et al 2001). For a review of Bayes factors, see Kass and Raftery (1995).

I am also interested in how Bayesian model averaging can be used as the basis for model-building strategies that take account of model uncertainty, providing an alternative to stepwise regression and related methods. My focus is on the practical implementation of these methods for model classes that arise in scientific applications, particularly in the social and health sciences. Hoeting et al (1999) give a review of Bayesian model averaging. For a discussion in the context of social science applications, which also exposits Bayes factors and the basis for the simple BIC approximation, see Raftery (1995). The Bayesian Model Averaging Homepage includes articles on BMA and free software for carrying it out.

Most recently, I have worked on extending Bayesian model averaging beyond statistical models to the dynamical deterministic simulation models that predominate in some environmental, engineering and policy-oriented disciplines. One motivation is to develop calibrated probabilistic weather forecasting methods using forecast ensembles (Raftery et al 2005).

Recommendations

If you're interested in learning about these topics, I would recommend starting with Raftery (1995), which aims to give an accessible introduction to all these topics, including a brief overview of Bayesian estimation. An introduction to Bayesian model averaging is given by Hoeting et al (1999). A summary of some of the main results in the area is given by Raftery and Zheng (2003); this has a fairly good set of references. Kass and Raftery (1995) give a review of Bayes factors for model selection and model comparison that's at a higher technical level than the previous ones. Raftery et al (2005) is a fairly accessible description of the application of BMA to dynamical models. Another approach to prediction from dynamic systems under model uncertainty is Dynamic Model Averaging (DMA) (Raftery et al 2010). For a comparison of some more recent methods, see Porwal and Raftery (2022)

Papers

Porwal, A. and Raftery, A.E. Effect of model space priors on statistical inference with model uncertainty. New England Journal of Statistics and Data Science, in press.

Porwal, A. and Raftery, A.E. (2022). Comparing methods for statistical inference with model uncertainty. Proceedings of the National Academy of Sciences, 119:2120737119.

Mulder, J. and Raftery, A.E. (2019). BIC extensions for order-constrained model selection. Sociological Methods and Research, Article Number: 0049124119882459.

Hernandez, B., Raftery, A.E., Pennington, S.R. and Parnell, A.C. (2018). Bayesian Additive Regression Trees using Bayesian Model Averaging. Statistics and Computing 28:869--890. Earlier version.

Hung, L.H., Shi, K., Wu, M., Young, W.C., Raftery, A.E. and Yeung, K.Y. (2017). fastBMA: Scalable Network Inference and Transitive Reduction. Gigascience 6:issue 10. PubMed.

Russell, N., Murphy, T.B. and Raftery, A.E. (2015). Bayesian model averaging in model-based clustering and density estimation. Technical Report no. 635, Department of Statistics, University of Washington. Also arXiv:1506.09035.

Young, W.C., Raftery, A.E. and Yeung, K.Y. (in press). A posterior probability approach for gene regulatory network inference in genetic perturbation data. Mathematical Biosciences and Engineering, to appear. Earlier version.

Onorante, L. and Raftery, A.E. (2016). Dynamic Model Averaging in Large Model Spaces. European Economic Review 81:2-14.

Fronczuk, M., Raftery, A.E. and Yeung, K.Y. (2015). CyNetworkBMA: a Cytoscape app for inferring gene regulatory networks. Source Code for Biology and Medicine 10:article 11.

Young, W.C., Raftery, A.E. and Yeung, K.Y. (2014). Fast Bayesian Inference for Gene Regulatory Networks Using ScanBMA. BMC Systems Biology, 8:article 47.

Lenkoski, A., Eicher, T.S. and Raftery, A.E. (2014). Two-Stage Bayesian Model Averaging in Endogenous Variable Models. Econometric Reviews, 33:122-151. Earlier version.

Sloughter, J.M., Gneiting, T. and Raftery, A.E. (2013) Probabilistic Wind Vector Forecasting using Ensembles and Bayesian Model Averaging. Monthly Weather Review, 141:2107-2119.

Lo, K., Raftery, A.E., Dombek, K., Zhu, J., Schadt, E.E., Bumgarner, R.E. and Yeung, K.Y. (2012). Integrating External Biological Knowledge in the Construction of Regulatory Networks from Time-series Expression Data. BMC Systems Biology 6: article 101.

McCormick, T.M., Raftery, A.E., Madigan, D. and Burd, R.S. (2012). Dynamic Logistic Regression and Dynamic Model Averaging for Binary Classification. Biometrics 68:23-30.

Yeung, K.Y., Dombek, K.M., Lo, K., Mittler, J.E., Zhu, J., Schadt, E.E., Bumgarner, R.E. and Raftery, A.E. (2011). Construction of regulatory networks using expression time-series data of a genotyped population. Proceedings of the National Academy of Sciences 108:19436-19441.

Kleiber, W., Raftery, A.E. and Gneiting, T. (2011). Geostatistical model averaging for locally calibrated probabilistic quantitative precipitation forecasting. Journal of the American Statistical Association 106:1291-1303.

Kleiber, W., Raftery, A.E., Baars, J., Gneiting, T., Mass, C.F. and Grimit, E.P. (2011). Locally Calibrated Probabilistic Temperature Forecasting Using Geostatistical Model Averaging and Local Bayesian Model Averaging. Monthly Weather Review 139:2630-2649.

Chmielecki, R.M. and A.E. Raftery (2011). Probabilistic Visibility Forecasting Using Bayesian Model Averaging. Monthly Weather Review 139:1626--1636.

Steele, R.J. and Raftery, A.E. (2010). Performance of Bayesian Model Selection Criteria for Gaussian Mixture Models. In Frontiers of Statistical Decision Making and Bayesian Analysis (edited by M.-H. Chen et al), pages 113-130, New York: Springer. Earlier version.

Raftery, A.E. and L. Bao. (2010). Estimating and Projecting Trends in HIV/AIDS Generalized Epidemics Using Incremental Mixture Importance Sampling. Biometrics 66:1162-1173.

Eicher, T., Papageorgiou, C. and Raftery, A.E. (2010). Determining Growth Determinants: Default Priors and Predictive Performance in Bayesian Model Averaging. Journal of Applied Econometrics 26:30-55.

Raftery, A.E., Karny, M., and Ettler, P. (2010). Online Prediction Under Model Uncertainty Via Dynamic Model Averaging: Application to a Cold Rolling Mill. Technometrics 52:52-66.

Sloughter, J.M., Gneiting, T. and Raftery, A.E. (2010). Probabilistic Wind Speed Forecasting using Ensembles and Bayesian Model Averaging. Journal of the American Statistical Association 105:25-35.

Murphy, T.B., Dean. N. and Raftery, A.E. (2010). Variable Selection and Updating In Model-Based Discriminant Analysis for High Dimensional Data with Food Authenticity Applications. Annals of Applied Statistics 4:396-421.

Dean, N. and Raftery, A.E. (2010). Latent Class Analysis Variable Selection. Annals of the Institute of Statistical Mathematics 62:11-35.

Fraley, C., Raftery, A.E. and Gneiting, T. (2010). Calibrating Multi-Model Forecast Ensembles with Exchangeable and Missing Members using Bayesian Model Averaging. Monthly Weather Review 138:190-202.

Bao, L., Gneiting, T., Grimit, E.P., Guttorp, P. and Raftery, A.E. (2010). Bias correction and Bayesian Model Averaging for ensemble forecasts of surface wind direction. Monthly Weather Review 138:1811-1821.

Gottardo, R. and Raftery, A.E. (2009). Bayesian Robust Variable and Transformation Selection: A Unified Approach. Canadian Journal of Statistics, 37:1-20.

Gottardo, R. and Raftery, A.E. (2009). Markov chain Monte Carlo with mixtures of singular distributions. Journal of Computational and Graphical Statistics 17:949-975.

Eicher, T.S., Lenkoski, A. and Raftery, A.E. (2009). Bayesian Model Averaging and Endogeneity Under Model Uncertainty: An Application to Development Determinants. Working Paper no. 94, Center for Statistics and the Social Sciences, University of Washington.

Oehler, V.G., Yeung, K.Y., Choi, Y.E., Bumgarner, R.E., Raftery, A.E. and Radich, J.P. (2009). The derivation of diagnostic markers of chronic myeloid leukemia progression from microarray data. Blood 114:3292-3298. [An applicadtion iterative BMA.]

Annest, A., Bumgarner, R.E., Raftery, A.E. and Yeung, K.Y. (2009). Iterative Bayesian Model Averaging: a method for the application of survival analysis to high-dimensional microarray data. BMC Bioinformatics 10, article 72.

Fraley, C., Raftery, A.E., Sloughter, J.M. and Gneiting, T. (2007). ``ensembleBMA: An R Package for Probabilistic Forecasting using Ensembles and Bayesian Model Averaging.'' Technical Report no. 516, Department of Statistics, University of Washington.

Berrocal, V., Raftery, A.E. and Gneiting, T. (2007). Combining Spatial Statistical and Ensemble Information in Probabilistic Weather Forecasts. Monthly Weather Review, 135, 1386-1402.

Wilson, L.J., Beauregard, S., Raftery, A.E. and Verret, R. (2007). Calibrated Surface Temperature Forecasts from the Canadian Ensemble Prediction System Using Bayesian Model Averaging (with Discussion). Monthly Weather Review, 135, 1364-1385. Discussion pages 4226-4236.

Sloughter, J.M., Raftery, A.E. and Gneiting, T. (2007). Probabilistic Quantitative Precipitation Forecasting Using Bayesian Model Averaging. Monthly Weather Review, 135, 3209-3220.

Raftery, A.E., Newton, M.A., Satagopan, J.M. and Krivitsky, P. (2007). Estimating the Integrated Likelihood via Posterior Simulation Using the Harmonic Mean Identity (with Discussion). In Bayesian Statistics 8 (edited by J.M. Bernardo et al.), pp. 1-45, Oxford University Press.

Gottardo, R. and Raftery, A.E. (2006). Bayesian Robust Variable and Transformation Selection: A Unified Approach. Technical Report no. 508, Department of Statistics, University of Washington.

Raftery, A.E. and Dean, N. (2006). Variable Selection for Model-Based Clustering. Journal of the American Statistical Assocation, 101, 168-178.

Gottardo, R., Raftery, A.E., Yeung, K.Y. and Bumgarner, R.E. (2006). Bayesian Robust Inference for Differential Gene Expression in cDNA Microarrays with Multiple Samples. Biometrics, 62, 10-18.

Steele, R., Raftery, A.E. and Emond, M. (2006). Computing Normalizing Constants for Finite Mixture Models via Incremental Mixture Importance Sampling (IMIS). Journal of Computational and Graphical Statistics, 15, 712-734.

Gottardo, R. and Raftery, A.E. (2006). Bayesian Robust Variable and Transformation Selection: A Unified Approach. Technical Report no. 508, Department of Statistics, University of Washington.

Yeung, K.Y., Bumgarner, R.E. and Raftery, A.E. (2005). ``Bayesian Model Averaging: Development of an improved multi-class, gene selection and classification tool for microarray data.'' Bioinformatics, 21(10), 2394-2402 (doi:10.1093/bioinformatics/bti319).

Raftery, A.E., Gneiting, T., Balabdaoui, F. and Polakowski, M. (2005). Using Bayesian Model Averaging to Calibrate Forecast Ensembles. Monthly Weather Review, 133, 1155-1174.

Walsh, D.C.I. and Raftery, A.E. (2005). Classification of mixtures of spatial point processes via partial Bayes factors. Journal of Computational and Graphical Statistics, 14, 139-154.

Gottardo, R. and Raftery, A.E. (2004). Markov chain Monte Carlo with mixtures of singular distributions. Technical Report no. 470, Department of Statistics, University of Washington.

Raftery, A.E. and Zheng, Y. (2003). Discussion: Performance of Bayesian Model Averaging. Journal of the American Statistical Association, 98, 931-938.

Stanford, D.C. and Raftery, A.E. (2002). Approximate Bayes factors for image segmentation: The Pseudolikelihood Information Criterion (PLIC). IEEE Transactions on Pattern Analysis and Machine Intelligence, 24, 1517-1520.

Hoeting, J.A., Raftery, A.E. and Madigan, D. (2002). Bayesian variable and transformation selection in linear regression. Journal of Computational and Graphical Statistics, 11, 485-507.

Oh, M.-S. and Raftery, A.E. (2001). Bayesian Multidimensional Scaling and Choice of Dimension. Journal of the American Statistical Association, 96, 1031-1044.

Viallefont, V., Raftery, A.E. and Richardson, S. (2001). Variable selection and Bayesian model averaging in epidemiological case-control studies. Statistics in Medicine, 20, 3215-3230.

Volinsky, C.T. and Raftery, A.E. (2000). Bayesian information criterion for censored survival models. Biometrics, 56, 256--262.

Hoeting, J.A., Madigan, D., Raftery, A.E. and Volinsky, C.T. (1999). Bayesian model averaging: A tutorial (with Discussion). Statistical Science, 14, 382--401. [Corrected version.] Correction: vol. 15, pp. 193-195. The corrected version is available at http://www.stat.washington.edu/www/research/online/hoeting1999.pdf. If cited, the corrected version should also be referenced, as here.

Volinsky, C.T., Madigan, D., Raftery, A.E. and Kronmal, R.A. (1997). Bayesian model averaging in proportional hazard models: Assessing stroke risk. Journal of the Royal Statistical Society, series C---Applied Statistics, 46, 433-448.

DiCiccio, T.J., Kass, R.E., Raftery, A.E. and Wasserman, L. (1997). Computing Bayes Factors by Combining Simulation and Asymptotic Approximations. Journal of the American Statistical Association, 92, 903-915.

Lewis, S.M. and Raftery, A.E. (1997) Estimating Bayes factors via posterior simulation with the Laplace-Metropolis estimator. Journal of the American Statistical Assocation, 92, 648-655.

Raftery, A.E., Madigan, D. and Hoeting, J.A. (1997). Bayesian model averaging for regression models. Journal of the American Statistical Association, 92, 179-191.

Madigan, D., Raftery, A.E., Volinsky, C.T., and Hoeting, J.A. (1996). Bayesian model averaging. In Integrating Multiple Learned Models, (IMLM-96), P. Chan, S. Stolfo, and D. Wolpert (Eds.), pp. 77-83.

Hoeting, J.A., Raftery, A.E. and Madigan, D. (1996). A method for simultaneous variable selection and outlier identification in linear regression. Computational Statistics and Data Analysis, 22, 251-270.

Raftery, A.E. (1996). Approximate Bayes factors and accounting for model uncertainty in generalized linear models. Biometrika, 83, 251-266.

Le, N.D., Raftery, A.E. and Martin, R.D. (1996). Robust order selection in autoregressive models using robust Bayes factors. Journal of the American Statistical Association, 91, 123-131.

Raftery, A.E. and Richardson, S. (1996). Model selection for generalized linear models via GLIB, with application to epidemiology. In Bayesian Biostatistics (D.A. Berry and D.K. Stangl, eds.), New York: Marcel Dekker, pp. 321--354. Earlier version (ps).

Raftery, A.E. (1996). Hypothesis testing and model selection. In Markov Chain Monte Carlo in Practice(W.R. Gilks, D.J. Spiegelhalter and S. Richardson, eds.), London: Chapman and Hall, pp. 163--188. Earlier version (ps).

Madigan, D., Gavrin, J. and Raftery, A.E. (1995). Enhancing the predictive performance of Bayesian graphical models. Communications in Statistics - Theory and Methods, 24, 2271-2292. Earlier technical report version (ps): Technical Report no. 270, Department of Statistics, University of Washington, February 1994.

Raftery, A.E., Madigan, D. and Volinsky, C.T. (1995). Accounting for model uncertainty in survival analysis improves predictive performance (with Discussion). In Bayesian Statistics 5 (J.M. Bernardo, J.O. Berger, A.P. Dawid and A.F.M. Smith, eds.), Oxford University Press, pp. 323-349. Earlier version (ps).

Raftery, A.E. (1995). Bayesian model selection in social research (with Discussion). Sociological Methodology, 25, 111-196.
Discussion: Avoiding model selection in Bayesian social research, by A. Gelman and D. B. Rubin.
Discussion: Better rules for better decisions, by R. M. Hauser.
Rejoinder: Model selection is unavoidable in social research, by A. E. Raftery.

Kass, R.E. and Raftery, A.E. (1995). Bayes factors. Journal of the American Statistical Association, 90, 773-795.

Madigan, D.M. and Raftery, A.E. (1994). Model selection and accounting for model uncertainty in graphical models using Occam's Window. Journal of the American Statistical Association, 89, 1335-1346.

Madigan, D., Raftery, A.E., York, J.C., Bradshaw, J.M., and Almond, R.G. (1993). Strategies for graphical model selection. Proceedings of the 4th International Workshop on Artificial Intelligence and Statistics, pp. 361-366. Earlier version (ps).

Raftery, A.E. (1993). Bayesian model selection in structural equation models. In Testing Stuctural Equation Models (K.A. Bollen and J.S. Long, eds.), Beverly Hills: Sage, pp. 163-180. Earlier version.

Raftery, A.E. (1989). Are ozone exceedance rates decreasing? Statistical Science, 4, 378-381.

Akman, V.E. and Raftery, A.E. (1986). Bayes factors for non-homogeneous Poisson processes with vague prior information. Journal of the Royal Statistical Society, series B, 48, 322-329.

Raftery, A.E. and Akman, V.E. (1986). Bayesian analysis of a Poisson process with a change-point. Biometrika, 73, 85-89.

Raftery, A.E. (1986). A note on Bayes factors for log-linear contingency table models with vague prior information. Journal of the Royal Statistical Society, series B, 48, 249-250.

Raftery, A.E. (1986). Choosing models for cross-classifications. American Sociological Review, 51, 145-146.

These papers are being made available here to facilitate the timely dissemination of scholarly work; copyright and all related rights are retained by the copyright holders.

Updated May 10, 2023.