Simulation and the Monte Carlo Method

Reuven Y. Rubinstein, DSc, was Professor Emeritus in the Faculty of Industrial Engineering and Management at Technion-Israel Institute of Technology. He served as a consultant at numerous large-scale organizations, such as IBM, Motorola, and NEC. The author of over 100 articles and six books, Dr. Rubinstein was also the inventor of the popular score-function method in simulation analysis and generic cross-entropy methods for combinatorial optimization and counting. Dirk P. Kroese, PhD, is a Professor of Mathematics and Statistics in the School of Mathematics and Physics of The University of Queensland, Australia. He has published over 100 articles and four books in a wide range of areas in applied probability and statistics, including Monte Carlo methods, cross-entropy, randomized algorithms, tele-traffic c theory, reliability, computational statistics, applied probability, and stochastic modeling.

Preface xiii Acknowledgments xvii 1 Preliminaries 1 1.1 Introduction 1 1.2 Random Experiments 1 1.3 Conditional Probability and Independence 2 1.4 Random Variables and Probability Distributions 4 1.5 Some Important Distributions 5 1.6 Expectation 6 1.7 Joint Distributions 7 1.8 Functions of Random Variables 11 1.8.1 Linear Transformations 12 1.8.2 General Transformations 13 1.9 Transforms 14 1.10 Jointly Normal Random Variables 15 1.11 Limit Theorems 16 1.12 Poisson Processes 17 1.13 Markov Processes 19 1.13.1 Markov Chains 19 1.13.2 Classification of States 21 1.13.3 Limiting Behavior 22 1.13.4 Reversibility 24 1.13.5 Markov Jump Processes 25 1.14 Gaussian Processes 27 1.15 Information 28 1.15.1 Shannon Entropy 29 1.15.2 KullbackLeibler Cross-Entropy 31 1.15.3 Maximum Likelihood Estimator and Score Function 32 1.15.4 Fisher Information 33 1.16 Convex Optimization and Duality 34 1.16.1 Lagrangian Method 35 1.16.2 Duality 37 Problems 41 References 46 2 Random Number, Random Variable, and Stochastic Process Generation 49 2.1 Introduction 49 2.2 Random Number Generation 49 2.2.1 Multiple Recursive Generators 51 2.2.2 Modulo 2 Linear Generators 52 2.3 Random Variable Generation 55 2.3.1 Inverse-Transform Method 55 2.3.2 Alias Method 57 2.3.3 Composition Method 58 2.3.4 AcceptanceRejection Method 59 2.4 Generating from Commonly Used Distributions 62 2.4.1 Generating Continuous Random Variables 62 2.4.2 Generating Discrete Random Variables 67 2.5 Random Vector Generation 70 2.5.1 Vector AcceptanceRejection Method 71 2.5.2 Generating Variables from a Multinormal Distribution 72 2.5.3 Generating Uniform Random Vectors over a Simplex 73 2.5.4 Generating Random Vectors Uniformly Distributed over a Unit Hyperball and Hypersphere 74 2.5.5 Generating Random Vectors Uniformly Distributed inside a Hyperellipsoid 75 2.6 Generating Poisson Processes 75 2.7 Generating Markov Chains and Markov Jump Processes 77 2.7.1 Random Walk on a Graph 78 2.7.2 Generating Markov Jump Processes 79 2.8 Generating Gaussian Processes 80 2.9 Generating Diffusion Processes 81 2.10 Generating Random Permutations 83 Problems 85 References 89 3 Simulation of Discrete-Event Systems 91 3.1 Introduction 91 3.2 Simulation Models 92 3.2.1 Classification of Simulation Models 94 3.3 Simulation Clock and Event List for DEDS 95 3.4 Discrete-Event Simulation 97 3.4.1 Tandem Queue 97 3.4.2 Repairman Problem 101 Problems 103 References 106 4 Statistical Analysis of Discrete-Event Systems 107 4.1 Introduction 107 4.2 Estimators and Confidence Intervals 108 4.3 Static Simulation Models 110 4.4 Dynamic Simulation Models 112 4.4.1 Finite-Horizon Simulation 114 4.4.2 Steady-State Simulation 114 4.5 Bootstrap Method 126 Problems 127 References 130 5 Controlling the Variance 133 5.1 Introduction 133 5.2 Common and Antithetic Random Variables 134 5.3 Control Variables 137 5.4 Conditional Monte Carlo 139 5.4.1 Variance Reduction for Reliability Models 141 5.5 Stratified Sampling 144 5.6 Multilevel Monte Carlo 146 5.7 Importance Sampling 149 5.7.1 Weighted Samples 149 5.7.2 Variance Minimization Method 150 5.7.3 Cross-Entropy Method 154 5.8 Sequential Importance Sampling 159 5.9 Sequential Importance Resampling 165 5.10 Nonlinear Filtering for Hidden Markov Models 167 5.11 Transform Likelihood Ratio Method 171 5.12 Preventing the Degeneracy of Importance Sampling 174 Problems 179 References 184 6 Markov Chain Monte Carlo 187 6.1 Introduction 187 6.2 MetropolisHastings Algorithm 188 6.3 Hit-and-Run Sampler 193 6.4 Gibbs Sampler 194 6.5 Ising and Potts Models 197 6.5.1 Ising Model 197 6.5.2 Potts Model 198 6.6 Bayesian Statistics 200 6.7 Other Markov Samplers 202 6.7.1 Slice Sampler 204 6.7.2 Reversible Jump Sampler 205 6.8 Simulated Annealing 208 6.9 Per