Introduction to Linear Regression Analysis

DOUGLAS C. MONTGOMERY, PHD, is Regents Professor of Industrial Engineering and Statistics at Arizona State University. Dr. Montgomery is the co-author of several Wiley books including Introduction to Linear Regression Analysis, 5th Edition.ELIZABETH A. PECK, PHD, is Logistics Modeling Specialist at the Coca-Cola Company in Atlanta, Georgia. G. GEOFFREY VINING, PHD, is Professor in the Department of Statistics at Virginia Polytechnic and State University. Dr. Peck is co-author of Introduction to Linear Regression Analysis, 5th Edition.

• Preface xiiiAbout the Companion Website xvi1. Introduction 11.1 Regression and Model Building 11.2 Data Collection 51.3 Uses of Regression 91.4 Role of the Computer 102. Simple Linear Regression 122.1 Simple Linear Regression Model 122.2 Least-Squares Estimation of the Parameters 132.3 Hypothesis Testing on the Slope and Intercept 222.4 Interval Estimation in Simple Linear Regression 292.5 Prediction of New Observations 332.6 Coefficient of Determination 352.7 A Service Industry Application of Regression 372.8 Does Pitching Win Baseball Games? 392.9 Using SAS and R for Simple Linear Regression 412.10 Some Considerations in the Use of Regression 442.11 Regression Through the Origin 462.12 Estimation by Maximum Likelihood 522.13 Case Where the Regressor x Is Random 533. Multiple Linear Regression 693.1 Multiple Regression Models 693.2 Estimation of the Model Parameters 723.3 Hypothesis Testing in Multiple Linear Regression 863.4 Confidence Intervals in Multiple Regression 993.5 Prediction of New Observations 1063.6 A Multiple Regression Model for the Patient Satisfaction Data 1063.7 Does Pitching and Defense Win Baseball Games? 1083.8 Using SAS and R for Basic Multiple Linear Regression 1103.9 Hidden Extrapolation in Multiple Regression 1113.10 Standardized Regression Coefficients 1153.11 Multicollinearity 1213.12 Why Do Regression Coefficients Have the Wrong Sign? 1234. Model Adequacy Checking 1344.1 Introduction 1344.2 Residual Analysis 1354.3 PRESS Statistic 1564.4 Detection and Treatment of Outliers 1574.5 Lack of Fit of the Regression Model 1615. Transformations and Weighting To Correct Model Inadequacies 1775.1 Introduction 1775.2 Variance-Stabilizing Transformations 1785.3 Transformations to Linearize the Model 1825.4 Analytical Methods for Selecting a Transformation 1885.5 Generalized and Weighted Least Squares 1945.6 Regression Models with Random Effects 2006. Diagnostics for Leverage and Influence 2176.1 Importance of Detecting Influential Observations 2176.2 Leverage 2186.3 Measures of Influence: Cook's D 2216.4 Measures of Influence: DFFITS and DFBETAS 2236.5 A Measure of Model Performance 2256.6 Detecting Groups of Influential Observations 2266.7 Treatment of Influential Observations 2267. Polynomial Regression Models 2307.1 Introduction 2307.2 Polynomial Models in One Variable 2307.3 Nonparametric Regression 2437.4 Polynomial Models in Two or More Variables 2497.5 Orthogonal Polynomials 2558. Indicator Variables 2688.1 General Concept of Indicator Variables 2688.2 Comments on the Use of Indicator Variables 2818.3 Regression Approach to Analysis of Variance 2839. Multicollinearity 2939.1 Introduction 2939.2 Sources of Multicollinearity 2949.3 Effects of Multicollinearity 2969.4 Multicollinearity Diagnostics 3009.5 Methods for Dealing with Multicollinearity 3119.6 Using SAS to Perform Ridge and Principal-Component Regression 33610. Variable Selection and Model Building 34210.1 Introduction 34210.2 Computational Techniques for Variable Selection 35310.3 Strategy for Variable Selection and Model Building 36710.4 Case Study: Gorman and Toman Asphalt Data Using SAS 37011. Validation of Regression Models 38811.1 Introduction 38811.2 Validation Techniques 38911.3 Data from Planned Experiments 40112. Introduction to Nonlinear Regression 40512.1 Linear and Nonlinear Regression Models 40512.2 Origins of Nonlinear Models 40712.3 Nonlinear Least Squares 41112.4 Transformation to a Linear Model 41312.5 Parameter Estimation in a Nonlinear System 41612.6 Statistical Inference in Nonlinear Regression 42512.7 Examples of Nonlinear Regression Models 42712.8 Using SAS and R 42813. Generalized Linear Models 44013.1 Introduction 44013.2 Logistic Regression Models 44113.3 Poisson Regression 46313.4 The Generalized Linear Model 46914. Regression Analysis of Time Series Data 49514.1 Introduction to Regression Models for Time Series Data 49514.2 Detecting Autocorrelation: The Durbin–Watson Test 49614.3 Estimating the Parameters in Time Series Regression Models 50115. Other Topics in the Use of Regression Analysis 52115.1 Robust Regression 52115.2 Effect of Measurement Errors in the Regressors 53215.3 Inverse Estimation—The Calibration Problem 53415.4 Bootstrapping in Regression 53815.5 Classification and Regression Trees (CART) 54515.6 Neural Networks 54715.7 Designed Experiments for Regression 549Appendix A. Statistical Tables 561Appendix B. Data Sets for Exercises 573Appendix C. Supplemental Technical Material 602C.1 Background on Basic Test Statistics 602C.2 Background from the Theory of Linear Models 605C.3 Important Results on SS R and SS Res 609C.4 Gauss-Markov Theorem, Var(ε) = σ 2 I 615C.5 Computational Aspects of Multiple Regression 617C.6 Result on the Inverse of a Matrix 618C.7 Development of the PRESS Statistic 619C.8 Development of S(i) 2 621C.9 Outlier Test Based on R-Student 622C.10 Independence of Residuals and Fitted Values 624C.11 Gauss–Markov Theorem, Var(ε) = V 625C.12 Bias in MSRes When the Model Is Underspecified 627C.13 Computation of Influence Diagnostics 628C.14 Generalized Linear Models 629Appendix D. Introduction to SAS 641D.1 Basic Data Entry 642D.2 Creating Permanent SAS Data Sets 646D.3 Importing Data from an EXCEL File 647D.4 Output Command 648D.5 Log File 648D.6 Adding Variables to an Existing SAS Data Set 650Appendix E. Introduction to R to Perform Linear Regression Analysis 651E.1 Basic Background on R 651E.2 Basic Data Entry 652E.3 Brief Comments on Other Functionality in R 654E.4 R Commander 655References 656Index 670