Statistics

DAVID W. SCOTT is the Noah Harding Professor of Statistics at Rice University in Houston, Texas. He is a Fellow of the ASA, IMS, AAAS, an elected member of the ISI and received the 2004 Army Wilks Award and the 2008 ASA Founder's Award. He was formerly the Editor of the Journal of Computational and Graphical Statistics and currently serves as Co-Editor of Wiley Interdisciplinary Reviews: Computational Statistics. He is also the author of Multivariate Density Estimation: Theory, Practice, and Visualization.

• Preface xiii1 Data Analysis and Understanding 11.1 Exploring the Distribution of Data 11.1.1 Pearson’s Father–Son Height Data 21.1.2 Lord Rayleigh’s Data 31.1.3 Discussion 41.2 Exploring Prediction Using Data 41.2.1 Body and Brain Weights of Land Mammals 51.2.2 Space Shuttle Flight 25 51.2.3 Pearson’s Father–Son Height Data Revisited 71.2.4 Discussion 8Problems 82 Classical Probability 112.1 Experiments with Equally Likely Outcomes 112.1.1 Simple Outcomes 122.1.2 Compound Events and Set Operations 122.2 Probability Laws 132.2.1 Union and Intersection of Events A and B 132.2.1.1 Case (i) 142.2.1.2 Cases (ii) and (iii) 142.2.1.3 Case (iv) 142.2.2 Conditional Probability 142.2.2.1 Definition of Conditional Probability 152.2.2.2 Conditional Probability With More Than Two Events 152.2.3 Independent Events 162.2.4 Bayes Theorem 172.2.5 Partitions and Total Probability 182.3 Counting Methods 192.3.1 With Replacement 202.3.2 Without Replacement (Permutations) 202.3.3 Without Replacement or Order (Combinations) 212.3.4 Examples 212.3.5 Extended Combinations (Multinomial) 222.4 Countable Sets: Implications as n → ∞ 222.4.1 Selecting Even or Odd Integers 222.4.2 Selecting Rational Versus Irrational Numbers 232.5 Kolmogorov’s Axioms 232.6 Reliability: Series Versus Parallel Networks 242.6.1 Series Network 242.6.2 Parallel Network 25Problems 263 Random Variables and Models Derived From Classical Probability and Postulates 273.1 Random Variables and Probability Distributions: Discrete Uniform Example 273.1.1 Toss of a Single Die 283.1.2 Toss of a Pair of Dice 283.2 The Univariate Probability Density Function: Continuous Uniform Example 303.2.1 Using the PDF to Compute Probabilities 323.2.2 Using the PDF to Compute Relative Odds 333.3 Summary Statistics: Central and Non-Central Moments 333.3.1 Expectation, Average, and Mean 343.3.2 Expectation as a Linear Operator 353.3.3 The Variance of a Random Variable 353.3.4 Standardized Random Variables 363.3.5 Higher Order Moments 373.3.6 Moment Generating Function 373.3.7 Measurement Scales and Units of Measurement 383.3.7.1 The Four Measurement Scales 383.3.7.2 Units of Measurement 393.4 Binomial Experiments 393.5 Waiting Time for a Success: Geometric PMF 423.6 Waiting Time for r Successes: Negative Binomial 433.7 Poisson Process and Distribution 433.7.1 Moments of the Poisson PMF 443.7.2 Examples 453.8 Waiting Time for Poisson Events: Negative Exponential PDF 453.9 The Normal Distribution (Also Known as the Gaussian Distribution) 463.9.1 Standard Normal Distribution 483.9.2 Sums of Independent Normal Random Variables 493.9.3 Normal Approximation to the Poisson Distribution 49Problems 504 Bivariate Random Variables, Transformations, and Simulations 514.1 Bivariate Continuous Random Variables 514.1.1 Joint CDF and PDF Functions 514.1.2 Marginal PDF 524.1.3 Conditional Probability Density Function 524.1.4 Independence of Two Random Variables 544.1.5 Expectation, Correlation, and Regression 544.1.5.1 Covariance and Correlation 554.1.5.2 Regression Function 554.1.6 Independence of n Random Variables 564.1.7 Bivariate Normal PDF 564.1.8 Correlation, Independence, and Confounding Variables 574.2 Change of Variables 574.2.1 Examples: Two Uniform Transformations 574.2.2 One-Dimensional Transformations 584.2.2.1 Example 1: Negative exponential PDF 594.2.2.2 Example 2: Cauchy PDF 604.2.2.3 Example 3: Chi-squared PDF with one degree of freedom 604.2.3 Two-Dimensional Transformations 604.3 Simulations 624.3.1 Generating Uniform Pseudo-Random Numbers 624.3.1.1 Reproducibility 624.3.1.2 RANDU 624.3.2 Probability Integral Transformation 634.3.3 Event-driven Simulation 63Problems 645 Approximations and Asymptotics 675.1 Why Do We Like Random Samples? 675.1.1 When u(X) Takes a Product Form 685.1.2 When u(X) Takes a Summation Form 685.2 Useful Inequalities 695.2.1 Markov’s Inequality 695.2.2 Chebyshev’s Inequality 705.2.3 Jensen’s Inequality 705.2.4 Cauchy–Schwarz Inequality 715.3 Sequences of Random Variables 725.3.1 Weak Law of Large Numbers 735.3.2 Consistency of the Sample Variance 735.3.3 Relationships Among the Modes of Convergence 745.3.3.1 Proof of Result (5.21) 745.3.3.2 Proof of Result (5.22) 745.4 Central Limit Theorem 755.4.1 Moment Generating Function for Sums 755.4.2 Standardizing the Sum Sn 755.4.3 Proof of Central Limit Theorem 765.5 Delta Method and Variance-stabilizing Transformations 77Problems 786 Parameter Estimation 796.1 Desirable Properties of an Estimator 806.2 Moments of the Sample Mean and Variance 806.2.1 Theoretical Mean and Variance of the Sample Mean 816.2.2 Theoretical Mean of the Sample Variance 816.2.3 Theoretical Variance of the Sample Variance 826.3 Method of Moments (MoM) 826.4 Sufficient Statistics and Data Compression 836.5 Bayesian Parameter Estimation 856.6 Maximum Likelihood Parameter Estimation 866.6.1 Relationship to Bayesian Parameter Estimation 876.6.2 Poisson MLE Example 876.6.3 Normal MLE Example 876.6.4 Uniform MLE Example 886.7 Information Inequalities and the Cramér–Rao Lower Bound 896.7.1 Score Function 896.7.2 Asymptotics of the MLE 906.7.3 Minimum Variance of Unbiased Estimators 916.7.4 Examples 91Problems 927 Hypothesis Testing 937.1 Setting up a Hypothesis Test 947.1.1 Example of a Critical Region 947.1.2 Accuracy and Errors in Hypothesis Testing 957.2 Best Critical Region for Simple Hypotheses 967.2.1 Simple Example Continued 967.2.2 Normal Shift Model with Common Variance 977.3 Best Critical Region for a Composite Alternative Hypothesis 987.3.1 Negative Exponential Composite Hypothesis Test 997.3.1.1 Example 997.3.1.2 Alternative Critical Regions 997.3.1.3 Mount St. Helens Example 1007.3.2 Normal Shift Model with Common But Unknown Variance: The T-test 1027.3.3 The Random Variable Tn−1 1027.3.3.1 Where We Show X and S2 are Independent 1027.3.3.2 Where We Show That S2 Scaled is ;;2(n − 1) 1037.3.3.3 Where We Finally Derive the T PDF 1037.3.4 The One-Sample T-test 1047.3.5 Example 1057.3.6 Other T-tests 1067.3.6.1 Paired T-test 1067.3.6.2 Two-Sample T-test 1077.3.6.3 Example Two-Sample T-test: Lord Rayleigh’s Data 1077.4 Reporting Results: p-values and Power 1087.4.1 Example When the Null Hypothesis Is Rejected 1097.4.2 When the Null Hypothesis is Not Rejected 1097.4.3 The Power Function 1107.5 Multiple Testing and the Bonferroni Correction 111Problems 1118 Confidence Intervals and Other Hypothesis Tests 1138.1 Confidence Intervals 1138.1.1 Confidence Interval for ;;: Normal Data, ;;2 Known 1138.1.2 Confidence Interval for ;;: ;;2 Unknown 1148.1.3 Confidence Intervals and p-values 1158.2 Hypotheses About the Variance and the F-Distribution 1158.2.1 The F-Distribution 1168.2.2 Hypotheses About the Value of the Variance 1168.2.3 Confidence Interval for the Variance 1178.2.4 Two-Sided Alternative for Testing ;;2 = ;;2 0 1178.3 Pearson’s Chi-Squared Tests 1188.3.1 The Multinomial PMF 1188.3.2 Goodness-of-Fit (GoF) Tests 1188.3.3 Two-Category Binomial Case 1198.3.4 m-Category Multinomial Case 1208.3.5 Goodness-of-Fit Test for a Parametric Model 1208.3.6 Tests for Independence in Contingency Tables 1228.4 Correlation Coefficient Tests and CIs 1238.4.1 How to Test if the Correlation ;; = 0 1238.4.2 Confidence Intervals and Tests for a General Correlation Coefficient 1258.5 Linear Regression 1258.5.1 Least Squares Regression 1258.5.2 Distribution of the Least-Squares Parameters 1268.5.3 A Confidence Interval for the Slope 1278.5.4 A Two-side Hypothesis Test for the Slope 1288.5.5 Predictions at a New Value 1288.5.6 Population Interval at a New Value 1288.6 Analysis of Variance 129Problems 1319 Topics in Statistics 1339.1 MSE and Histogram Bin Width Selection 1339.1.1 MSE Criterion for Biased Estimators 1339.1.2 Case Study: Optimal Histogram Bin Widths 1349.1.3 Examples with Normal Data 1379.1.4 Normal Reference Rules for the Histogram Bin Width 1379.1.4.1 Scott’s Rule 1379.1.4.2 Freedman–Diaconis Rule 1379.1.4.3 Sturges’ Rule 1389.1.4.4 Comparison of the Three Rules 1389.2 An Optimal Stopping Time Problem 1399.3 Compound Random Variables 1419.3.1 Computing Expectations with Conditioning 1419.3.2 Sum of a Random Number of Random Variables 1429.4 Simulation and the Bootstrap 1439.5 Multiple Linear Regression 1449.6 Experimental Design 1459.7 Logistic Regression, Poisson Regression, and the Generalized Linear Model 1479.8 Robustness 1489.9 Conclusions 150Appendices 151A Notation Used in This Book 151B Common Distributions 153C Using R and Mathematica For This Text 154C.1 R Language – The Very Basics 154C.2 Mathematica – The Basics 155Bibliography 157Index 159