Simplicity, Complexity and Modelling

Stephen Senn Professor of Statistics, Department of Statistics, University of Glasgow. Senior Professor and Editor of the Wiley Statistics in Practice Series. His research concerns statistics applied to drug development, a subject he has written many papers on. Philip Dawid is a Professor in Statistics at Cambridge University. Researching fundamentally into the logical foundation of probability and statistics in terms of forecasting. He is currently an Editor of the journal Bayesian Analysis. Mike Christie is Professor of Reservoir Engineering at Heriot-Way University. His research interests lie in accurate numerical modeling of fluid flow, in porous media primarily. Andrew Cliffe Professor of Computational Applied Mathematics at the University of Nottingham. His research interests include fluid dynamics and nuclear waste disposal.

“In short, this book offers plenty.  While reading it cannot entirely replace first-hand experience of actually working with statistical modelling, I think it can be highly useful, either for a course on Ph.D. level, or for a statistician setting out on her own to improve her competence in applying statistical techniques and modelling in non-trivial situations. (International Statistical Review, 1 December 2012)

• Preface ix Acknowledgements xiContributing authors xiii1 Introduction 1Mike Christie, Andrew Cliffe, Philip Dawid and Stephen Senn1.1 The origins of the SCAM project 11.2 The scope of modelling in the modern world 21.3 The different professions and traditions engaged in modelling 31.4 Different types of models 31.5 Different purposes for modelling 51.6 The purpose of the book 61.7 Overview of the chapters 6References 82 Statistical model selection 11Philip Dawid and Stephen Senn2.1 Introduction 112.2 Explanation or prediction? 122.3 Levels of uncertainty 122.4 Bias–variance trade-off 132.5 Statistical models 152.5.1 Within-model inference 162.6 Model comparison 182.7 Bayesian model comparison 182.7.1 Model uncertainty 192.7.2 Laplace approximation 202.8 Penalized likelihood 202.8.1 Bayesian information criterion 212.9 The Akaike information criterion 212.9.1 Inconsistency of AIC 232.10 Significance testing 232.11 Many variables 272.12 Data-driven approaches 282.12.1 Cross-validation 292.12.2 Prequential analysis 292.13 Model selection or model averaging? 30References 313 Modelling in drug development 35Stephen Senn3.1 Introduction 353.2 The nature of drug development and scope for statistical modelling 363.3 Simplicity versus complexity in phase III trials 363.3.1 The nature of phase III trials 363.3.2 The case for simplicity in analysing phase III trials 373.3.3 The case for complexity in modelling clinical trials 383.4 Some technical issues 393.4.1 The effect of covariate adjustment in linear models 403.4.2 The effect of covariate adjustment in non-linear models 423.4.3 Random effects in multi-centre trials 443.4.4 Subgroups and interactions 453.4.5 Bayesian approaches 463.5 Conclusion 463.6 Appendix: The effect of covariate adjustment on the variance multiplier in least squares 47References 484 Modelling with deterministic computer models 51Jeremy E. Oakley4.1 Introduction 514.2 Metamodels and emulators for computationally expensive simulators 524.2.1 Gaussian processes emulators 534.2.2 Multivariate outputs 564.3 Uncertainty analysis 574.4 Sensitivity analysis 584.4.1 Variance-based sensitivity analysis 584.4.2 Value of information 614.5 Calibration and discrepancy 634.6 Discussion 64References 655 Modelling future climates 69Peter Challenor and Robin Tokmakian5.1 Introduction 695.2 What is the risk from climate change? 705.3 Climate models 705.4 An anatomy of uncertainty 725.4.1 Aleatoric uncertainty 725.4.2 Epistemic uncertainty 735.5 Simplicity and complexity 755.6 An example: The collapse of the thermohaline circulation 775.7 Conclusions 79References 796 Modelling climate change impacts for adaptation assessments 83Suraje Dessai and Jeroen van der Sluijs6.1 Introduction 836.1.1 Climate impact assessment 846.2 Modelling climate change impacts: From world development paths to localized impacts 876.2.1 Greenhouse gas emissions 876.2.2 Climate models 906.2.3 Downscaling 936.2.4 Regional/local climate change impacts 946.3 Discussion 956.3.1 Multiple routes of uncertainty assessment 966.3.2 What is the appropriate balance between simplicity and complexity? 96References 987 Modelling in water distribution systems 103Zoran Kapelan7.1 Introduction 1037.2 Water distribution system models 1047.2.1 Water distribution systems 1047.2.2 WDS hydraulic models 1047.2.3 Uncertainty in WDS hydraulic modelling 1077.3 Calibration of WDS hydraulic models 1087.3.1 Calibration problem 1087.3.2 Existing approaches 1097.3.3 Case study 1137.4 Sampling design for calibration 1167.4.1 Sampling design problem 1167.4.2 Existing approaches 1167.4.3 Case study 1207.5 Summary and conclusions 120References 1228 Modelling for flood risk management 125Jim Hall8.1 Introduction 1258.2 Flood risk management 1268.2.1 Long-term change 1308.2.2 Uncertainty 1318.3 Multi-purpose management 1318.4 Modelling for flood risk management 1328.4.1 Source 1328.4.2 Pathway 1328.4.3 Receptors 1358.4.4 An example of a system model: Towyn 1358.5 Model choice 1378.6 Conclusions 143References 1449 Uncertainty quantification and oil reservoir modelling 147Mike Christie9.1 Introduction 1479.2 Bayesian framework 1489.2.1 Solution errors 1499.3 Quantifying uncertainty in prediction of oil recovery 1509.3.1 Stochastic sampling algorithms 1519.3.2 Computing uncertainties from multiple history matched models 1539.4 Inverse problems and reservoir model history matching 1559.4.1 Synthetic problems 1559.4.2 Imperial college fault model 1579.4.3 Comparison of algorithms on a real field example 1589.5 Selecting appropriate detail in models 1629.5.1 Adaptive multiscale estimation 1629.5.2 Bayes factors 1659.5.3 Application of solution error modelling 1679.6 Summary 170References 17110 Modelling in radioactive waste disposal 173Andrew Cliffe10.1 Introduction 17310.2 The radioactive waste problem 17410.2.1 What is radioactive waste? 17410.2.2 How much radioactive waste is there? 17510.2.3 What are the options for long-term management of radioactive waste? 17510.3 The treatment of uncertainty in radioactive waste disposal 17710.3.1 Deep geological disposal 17710.3.2 Repository performance assessment 17710.3.3 Modelling 17910.3.4 Model verification and validation 18010.3.5 Strategies for dealing with uncertainty 18210.4 Summary and conclusions 184References 18411 Issues for modellers 187Mike Christie, Andrew Cliffe, Philip Dawid and Stephen Senn11.1 What are models and what are they useful for? 18711.2 Appropriate levels of complexity 18911.3 Uncertainty 19011.3.1 Model inputs and parameter uncertainty 19011.3.2 Model uncertainty 191References 192Glossary 193Index 201