Case Studies in Bayesian Statistical Modelling and Analysis

Clair Alston, Queensland University of Technology and Science, Australia. Kerrie L. Mengersen, Queensland University of Technology and Science, Australia. Tony Pettitt, Queensland University of Technology and Science, Australia.

“As such, this book can serve as a handy reference for proficient statisticians and programmers.”  (The Quarterly Review of Biology, 1 October 2015)

• Preface xviiList of contributors xix1 Introduction 1Clair L. Alston, Margaret Donald, Kerrie L. Mengersen and Anthony N. Pettitt1.1 Introduction 11.2 Overview 11.3 Further reading 81.3.1 Bayesian theory and methodology 81.3.2 Bayesian methodology 101.3.3 Bayesian computation 101.3.4 Bayesian software 111.3.5 Applications 13References 132 Introduction to MCMC 17Anthony N. Pettitt and Candice M. Hincksman2.1 Introduction 172.2 Gibbs sampling 182.2.1 Example: Bivariate normal 182.2.2 Example: Change-point model 192.3 Metropolis–Hastings algorithms 192.3.1 Example: Component-wise MH or MH within Gibbs 202.3.2 Extensions to basic MCMC 212.3.3 Adaptive MCMC 222.3.4 Doubly intractable problems 222.4 Approximate Bayesian computation 242.5 Reversible jump MCMC 252.6 MCMC for some further applications 26References 273 Priors: Silent or active partners of Bayesian inference? 30Samantha Low Choy3.1 Priors in the very beginning 303.1.1 Priors as a basis for learning 323.1.2 Priors and philosophy 323.1.3 Prior chronology 333.1.4 Pooling prior information 343.2 Methodology I: Priors defined by mathematical criteria 353.2.1 Conjugate priors 353.2.2 Impropriety and hierarchical priors 373.2.3 Zellner’s g-prior for regression models 373.2.4 Objective priors 383.3 Methodology II: Modelling informative priors 403.3.1 Informative modelling approaches 403.3.2 Elicitation of distributions 423.4 Case studies 443.4.1 Normal likelihood: Time to submit research dissertations 443.4.2 Binomial likelihood: Surveillance for exotic plant pests 473.4.3 Mixture model likelihood: Bioregionalization 503.4.4 Logistic regression likelihood: Mapping species distribution via habitat models 533.5 Discussion 573.5.1 Limitations 573.5.2 Finding out about the problem 583.5.3 Prior formulation 593.5.4 Communication 603.5.5 Conclusion 61Acknowledgements 61References 614 Bayesian analysis of the normal linear regression model 66Christopher M. Strickland and Clair L. Alston4.1 Introduction 664.2 Case studies 674.2.1 Case study 1: Boston housing data set 674.2.2 Case study 2: Production of cars and station wagons 674.3 Matrix notation and the likelihood 674.4 Posterior inference 684.4.1 Natural conjugate prior 694.4.2 Alternative prior specifications 734.4.3 Generalizations of the normal linear model 744.4.4 Variable selection 784.5 Analysis 814.5.1 Case study 1: Boston housing data set 814.5.2 Case study 2: Car production data set 85References 885 Adapting ICU mortality models for local data: A Bayesian approach 90Petra L. Graham, Kerrie L. Mengersen and David A. Cook5.1 Introduction 905.2 Case study: Updating a known risk-adjustment model for local use 915.3 Models and methods 925.4 Data analysis and results 965.4.1 Updating using the training data 965.4.2 Updating the model yearly 985.5 Discussion 100References 1016 A Bayesian regression model with variable selection for genome-wide association studies 103Carla Chen, Kerrie L. Mengersen, Katja Ickstadt and Jonathan M. Keith6.1 Introduction 1036.2 Case study: Case–control of Type 1 diabetes 1046.3 Case study: GENICA 1056.4 Models and methods 1056.4.1 Main effect models 1056.4.2 Main effects and interactions 1086.5 Data analysis and results 1096.5.1 WTCCC TID 1096.5.2 GENICA 1106.6 Discussion 112Acknowledgements 115References 1156.A  Appendix: SNP IDs 1177 Bayesian meta-analysis 118Jegar O. Pitchforth and Kerrie L. Mengersen7.1 Introduction 1187.2 Case study 1: Association between red meat consumption and breast cancer 1197.2.1 Background 1197.2.2 Meta-analysis models 1217.2.3 Computation 1257.2.4 Results 1257.2.5 Discussion 1297.3 Case study 2: Trends in fish growth rate and size 1307.3.1 Background 1307.3.2 Meta-analysis models 1317.3.3 Computation 1347.3.4 Results 1347.3.5 Discussion 135Acknowledgements 137References 1388 Bayesian mixed effects models 141Clair L. Alston, Christopher M. Strickland, Kerrie L. Mengersen and Graham E. Gardner8.1 Introduction 1418.2 Case studies 1428.2.1 Case study 1: Hot carcase weight of sheep carcases 1428.2.2 Case study 2: Growth of primary school girls 1428.3 Models and methods 1468.3.1 Model for Case study 1 1478.3.2 Model for Case study 2 1488.3.3 MCMC estimation 1498.4 Data analysis and results 1508.5 Discussion 158References 1589 Ordering of hierarchies in hierarchical models: Bone mineral density estimation 159Cathal D. Walsh and Kerrie L. Mengersen9.1 Introduction 1599.2 Case study 1609.2.1 Measurement of bone mineral density 1609.3 Models 1619.3.1 Hierarchical model 1629.3.2 Model H1 1639.3.3 Model H2 1639.4 Data analysis and results 1649.4.1 Model H1 1649.4.2 Model H2 1659.4.3 Implication of ordering 1669.4.4 Simulation study 1669.4.5 Study design 1669.4.6 Simulation study results 1679.5 Discussion 168References 1689.A  Appendix: Likelihoods 17010 Bayesian Weibull survival model for gene expression data 171Sri Astuti Thamrin, James M. McGree and Kerrie L. Mengersen10.1 Introduction 17110.2 Survival analysis 17210.3 Bayesian inference for the Weibull survival model 17410.3.1 Weibull model without covariates 17410.3.2 Weibull model with covariates 17510.3.3 Model evaluation and comparison 17610.4 Case study 17810.4.1 Weibull model without covariates 17810.4.2 Weibull survival model with covariates 18010.4.3 Model evaluation and comparison 18210.5 Discussion 182References 18311 Bayesian change point detection in monitoring clinical outcomes 186Hassan Assareh, Ian Smith and Kerrie L. Mengersen11.1 Introduction 18611.2 Case study: Monitoring intensive care unit outcomes 18711.3 Risk-adjusted control charts 18711.4 Change point model 18811.5 Evaluation 18911.6 Performance analysis 19011.7 Comparison of Bayesian estimator with other methods 19411.8 Conclusion 194References 19512 Bayesian splines 197Samuel Clifford and Samantha Low Choy12.1 Introduction 19712.2 Models and methods 19712.2.1 Splines and linear models 19712.2.2 Link functions 19812.2.3 Bayesian splines 19812.2.4 Markov chain Monte Carlo 20412.2.5 Model choice 20612.2.6 Posterior diagnostics 20712.3 Case studies 20712.3.1 Data 20712.3.2 Analysis 20812.4 Conclusion 21612.4.1 Discussion 21612.4.2 Extensions 21712.4.3 Summary 218References 21813 Disease mapping using Bayesian hierarchical models 221Arul Earnest, Susanna M. Cramb and Nicole M. White13.1 Introduction 22113.2 Case studies 22413.2.1 Case study 1: Spatio-temporal model examining the incidence of birth defects 22413.2.2 Case study 2: Relative survival model examining survival from breast cancer 22513.3 Models and methods 22513.3.1 Case study 1 22513.3.2 Case study 2 22913.4 Data analysis and results 23013.4.1 Case study 1 23013.4.2 Case study 2 23113.5 Discussion 234References 23714 Moisture, crops and salination: An analysis of a three-dimensional agricultural data set 240Margaret Donald, Clair L. Alston, Rick Young and Kerrie L. Mengersen14.1 Introduction 24014.2 Case study 24114.2.1 Data 24214.2.2 Aim of the analysis 24214.3 Review 24314.3.1 General methodology 24314.3.2 Computations 24314.4 Case study modelling 24314.4.1 Modelling framework 24314.5 Model implementation: Coding considerations 24614.5.1 Neighbourhood matrices and CAR models 24614.5.2 Design matrices vs indexing 24614.6 Case study results 24714.7 Conclusions 249References 25015 A Bayesian approach to multivariate state space modelling: A study of a Fama–French asset-pricing model with time-varying regressors 252Christopher M. Strickland and Philip Gharghori15.1 Introduction 25215.2 Case study: Asset pricing in financial markets 25315.2.1 Data 25415.3 Time-varying Fama–French model 25415.3.1 Specific models under consideration 25515.4 Bayesian estimation 25615.4.1 Gibbs sampler 25615.4.2 Sampling Σε 25715.4.3 Sampling β 1:n 25715.4.4 Sampling Σ  α 25915.4.5 Likelihood calculation 26015.5 Analysis 26115.5.1 Prior elicitation 26115.5.2 Estimation output 26115.6 Conclusion 264References 26516 Bayesian mixture models: When the thing you need to know is the thing you cannot measure 267Clair L. Alston, Kerrie L. Mengersen and Graham E. Gardner16.1 Introduction 26716.2 Case study: CT scan images of sheep 26816.3 Models and methods 27016.3.1 Bayesian mixture models 27016.3.2 Parameter estimation using the Gibbs sampler 27316.3.3 Extending the model to incorporate spatial information 27416.4 Data analysis and results 27616.4.1 Normal Bayesian mixture model 27616.4.2 Spatial mixture model 27816.4.3 Carcase volume calculation 28116.5 Discussion 284References 28417 Latent class models in medicine 287Margaret Rolfe, Nicole M. White and Carla Chen17.1 Introduction 28717.2 Case studies 28817.2.1 Case study 1: Parkinson’s disease 28817.2.2 Case study 2: Cognition in breast cancer 28817.3 Models and methods 28917.3.1 Finite mixture models 29017.3.2 Trajectory mixture models 29217.3.3 Goodness of fit 29617.3.4 Label switching 29717.3.5 Model computation 29817.4 Data analysis and results 30017.4.1 Case study 1: Phenotype identification in PD 30017.4.2 Case study 2: Trajectory groups for verbal memory 30217.5 Discussion 306References 30718 Hidden Markov models for complex stochastic processes: A case study in electrophysiology 310Nicole M. White, Helen Johnson, Peter Silburn, Judith Rousseau and Kerrie L. Mengersen18.1 Introduction 31018.2 Case study: Spike identification and sorting of extracellular recordings 31118.3 Models and methods 31218.3.1 What is an HMM? 31218.3.2 Modelling a single AP: Application of a simple HMM 31318.3.3 Multiple neurons: An application of a factorial HMM 31518.3.4 Model estimation and inference 31718.4 Data analysis and results 32018.4.1 Simulation study 32018.4.2 Case study: Extracellular recordings collected during deep brain stimulation 32318.5 Discussion 326References 32719 Bayesian classification and regression trees 330Rebecca A. O’Leary, Samantha Low Choy, Wenbiao Hu and Kerrie L. Mengersen19.1 Introduction 33019.2 Case studies 33219.2.1 Case study 1: Kyphosis 33219.2.2 Case study 2: Cryptosporidium 33219.3 Models and methods 33419.3.1 CARTs 33419.3.2 Bayesian CARTs 33519.4 Computation 33719.4.1 Building the BCART model – stochastic search 33719.4.2 Model diagnostics and identifying good trees 33919.5 Case studies – results 34119.5.1 Case study 1: Kyphosis 34119.5.2 Case study 2: Cryptosporidium 34319.6 Discussion 345References 34620 Tangled webs: Using Bayesian networks in the fight against infection 348Mary Waterhouse and Sandra Johnson20.1 Introduction to Bayesian network modelling 34820.1.1 Building a BN 34920.2 Introduction to case study 35120.3 Model 35220.4 Methods 35420.5 Results 35520.6 Discussion 357References 35921 Implementing adaptive dose finding studies using sequential Monte Carlo 361James M. McGree, Christopher C. Drovandi and Anthony N. Pettitt21.1 Introduction 36121.2 Model and priors 36321.3 SMC for dose finding studies 36421.3.1 Importance sampling 36421.3.2 SMC 36521.3.3 Dose selection procedure 36721.4 Example 36921.5 Discussion 371References 37221.A Appendix: Extra example 37322 Likelihood-free inference for transmission rates of nosocomial pathogens 374Christopher C. Drovandi and Anthony N. Pettitt22.1 Introduction 37422.2 Case study: Estimating transmission rates of nosocomial pathogens 37522.2.1 Background 37522.2.2 Data 37622.2.3 Objective 37622.3 Models and methods 37622.3.1 Models 37622.3.2 Computing the likelihood 37922.3.3 Model simulation 38022.3.4 ABC 38122.3.5 ABC algorithms 38222.4 Data analysis and results 38422.5 Discussion 385References 38623 Variational Bayesian inference for mixture models 388Clare A. McGrory23.1 Introduction 38823.2 Case study: Computed tomography (CT) scanning of a loin portion of a pork carcase 39023.3 Models and methods 39223.4 Data analysis and results 39723.5 Discussion 399References 39923.A Appendix: Form of the variational posterior for a mixture of multivariate normal densities 40124 Issues in designing hybrid algorithms 403Jeong E. Lee, Kerrie L. Mengersen and Christian P. Robert24.1 Introduction 40324.2 Algorithms and hybrid approaches 40624.2.1 Particle system in the MCMC context 40724.2.2 MALA 40724.2.3 DRA 40824.2.4 PS 40924.2.5 Population Monte Carlo (PMC) algorithm 41024.3 Illustration of hybrid algorithms 41224.3.1 Simulated data set 41224.3.2 Application: Aerosol particle size 41524.4 Discussion 417References 41825 A Python package for Bayesian estimation using Markov chain Monte Carlo 421Christopher M. Strickland, Robert J. Denham, Clair L. Alston and Kerrie L. Mengersen25.1 Introduction 42125.2 Bayesian analysis 42325.2.1 MCMC methods and implementation 42425.2.2 Normal linear Bayesian regression model 43325.3 Empirical illustrations 43725.3.1 Example 1: Linear regression model – variable selection and estimation 43825.3.2 Example 2: Loglinear model 44125.3.3 Example 3: First-order autoregressive regression 44625.4 Using PyMCMC efficiently 45125.4.1 Compiling code in Windows 45525.5 PyMCMC interacting with R 45725.6 Conclusions 45825.7 Obtaining PyMCMC 459References 459Index 461