Engineering Design via Surrogate Modelling

Dr. Alexander I. J. Forrester is Lecturer in Engineering Design at the University of Southampton. His main area of research focuses on improving the efficiency with which expensive analysis (particularly computational fluid dynamics) is used in design. His techniques have been applied to wing aerodynamics, satellite structures, sports equipment design and Formula One. Dr Andras Sobester is a Lecturer and EPSRC/ Royal Academy of Engineering research Fellow in the School of Engineering Sciences at the University of Southampton. His research interests include aircraft design, aerodynamic shape parameterization and optimization, as well as engineering design technology in general. Professor Andy J. Keane currently holds the Chair of Computational Engineering at the University of Southampton. He leads the University's Computational Engineering at the Design Research Group and directs the rolls-Royce University Technology centre for Computational Engineering. His interests lie primarily in the aerospace sciences, with a focus on the design of aerospace systems using computational methods. He has published over two hundred papers and three books in this area, many of which deal with surrogate modelling concepts.

Preface ix About the Authors xi Foreword xiii Prologue xv Part I Fundamentals 1 1 Sampling Plans 3 1.1 The Curse of Dimensionality and How to Avoid It 4 1.2 Physical versus Computational Experiments 4 1.3 Designing Preliminary Experiments (Screening) 6 1.3.1 Estimating the Distribution of Elementary Effects 6 1.4 Designing a Sampling Plan 13 1.4.1 Stratification 13 1.4.2 Latin Squares and Random Latin Hypercubes 15 1.4.3 Space-filling Latin Hypercubes 17 1.4.4 Space-filling Subsets 28 1.5 A Note on Harmonic Responses 29 1.6 Some Pointers for Further Reading 30 References 31 2 Constructing a Surrogate 33 2.1 The Modelling Process 33 2.1.1 Stage One: Preparing the Data and Choosing a Modelling Approach 33 2.1.2 Stage Two: Parameter Estimation and Training 35 2.1.3 Stage Three: Model Testing 36 2.2 Polynomial Models 40 2.2.1 Example One: Aerofoil Drag 42 2.2.2 Example Two: A Multimodal Testcase 44 2.2.3 What About the k-variable Case? 45 2.3 Radial Basis Function Models 45 2.3.1 Fitting Noise-Free Data 45 2.3.2 Radial Basis Function Models of Noisy Data 49 2.4 Kriging 49 2.4.1 Building the Kriging Model 51 2.4.2 Kriging Prediction 59 2.5 Support Vector Regression 63 2.5.1 The Support Vector Predictor 64 2.5.2 The Kernel Trick 67 2.5.3 Finding the Support Vectors 68 2.5.4 Finding 70 2.5.5 Choosing C and 71 2.5.6 Computing : v-SVR 73 2.6 The Big(ger) Picture 75 References 76 3 Exploring and Exploiting a Surrogate 77 3.1 Searching the Surrogate 78 3.2 Infill Criteria 79 3.2.1 Prediction Based Exploitation 79 3.2.2 Error Based Exploration 84 3.2.3 Balanced Exploitation and Exploration 85 3.2.4 Conditional Likelihood Approaches 91 3.2.5 Other Methods 101 3.3 Managing a Surrogate Based Optimization Process 102 3.3.1 Which Surrogate for What Use? 102 3.3.2 How Many Sample Plan and Infill Points? 102 3.3.3 Convergence Criteria 103 3.4 Search of the Vibration Isolator Geometry Feasibility Using Kriging Goal Seeking 104 References 106 Part II Advanced Concepts 109 4 Visualization 111 4.1 Matrices of Contour Plots 112 4.2 Nested Dimensions 114 Reference 116 5 Constraints 117 5.1 Satisfaction of Constraints by Construction 117 5.2 Penalty Functions 118 5.3 Example Constrained Problem 121 5.3.1 Using a Kriging Model of the Constraint Function 121 5.3.2 Using a Kriging Model of the Objective Function 123 5.4 Expected Improvement Based Approaches 125 5.4.1 Expected Improvement with Simple Penalty Function 126 5.4.2 Constrained Expected Improvement 126 5.5 Missing Data 131 5.5.1 Imputing Data for Infeasible Designs 133 5.6 Design of a Helical Compression Spring Using Constrained Expected Improvement 136 5.7 Summary 139 References 139 6 Infill Criteria with Noisy Data 141 6.1 Regressing Kriging 143 6.2 Searching the Regression Model 144 6.2.1 Re-Interpolation 146 6.2.2 Re-Interpolation with Conditional Likelihood Approaches 149 6.3 A Note on Matrix Ill-Conditioning 152 6.4 Summary 152 References 153 7 Exploiting Gradient Information 155 7.1 Obtaining Gradients 155 7.1.1 Finite Differencing 155 7.1.2 Complex Step Approximation 156 7.1.3 Adjoint Methods and Algorithmic Differentiation 156 7.2 Gradient-enhanced Modelling 157 7.3 Hessian-enhanced Modelling 162 7.4 Summary 165 References 165 8 Multi-fidelity Analysis 167 8.1 Co-Kriging 167 8.2 One-variable Demonstration 173 8.3 Choosing Xc and Xe 176 8.4 Summary 177 References 177 9 Multiple Design Objectives 179 9.1 Pareto Optimization 179 9.2 Multi-objective Expected Improvement 182 9.3 Design of the Nowacki Cantilever Beam Using Multi-objective, Constrained Expected Improvement 186 9.4 Design of a Helical Compression Spring Using Multi-objective, Constrained Expected Improvement 191 9.5 Summary 192 References 192 Appendix: Example Problems 195 A.1 One-Variable Test Function 1