Handbook of Regression Analysis With Applications in R

Samprit Chatterjee, PhD, is Professor Emeritus of Statistics at New York University. A Fellow of the American Statistical Association, Dr. Chatterjee has been a Fulbright scholar in both Kazakhstan and Mongolia. He is the coauthor of multiple editions of Regression Analysis By Example, Sensitivity Analysis in Linear Regression, A Casebook for a First Course in Statistics and Data Analysis, and the first edition of Handbook of Regression Analysis, all published by Wiley. Jeffrey S. Simonoff, PhD, is Professor of Statistics at the Leonard N. Stern School of Business of New York University. He is a Fellow of the American Statistical Association, a Fellow of the Institute of Mathematical Statistics, and an Elected Member of the International Statistical Institute. He has authored, coauthored, or coedited more than one hundred articles and seven books on the theory and applications of statistics.

• Preface to the Second Edition xvPreface to the First Edition xixPart I The Multiple Linear Regression Model1 Multiple Linear Regression 31.1 Introduction 31.2 Concepts and Background Material 41.2.1 The Linear Regression Model 41.2.2 Estimation Using Least Squares 51.2.3 Assumptions 81.3 Methodology 91.3.1 Interpreting Regression Coefficients 91.3.2 Measuring the Strength of the Regression Relationship 101.3.3 Hypothesis Tests and Confidence Intervals for β 121.3.4 Fitted Values and Predictions 131.3.5 Checking Assumptions Using Residual Plots 141.4 Example —Estimating Home Prices 151.5 Summary 192 Model Building 232.1 Introduction 232.2 Concepts and Background Material 242.2.1 Using Hypothesis Tests to Compare Models 242.2.2 Collinearity 262.3 Methodology 292.3.1 Model Selection 292.3.2 Example—Estimating Home Prices (continued) 312.4 Indicator Variables and Modeling Interactions 382.4.1 Example—Electronic Voting and the 2004 Presidential Election 402.5 Summary 46Part II Addressing Violations of Assumptions3 Diagnostics for Unusual Observations 533.1 Introduction 533.2 Concepts and Background Material 543.3 Methodology 563.3.1 Residuals and Outliers 563.3.2 Leverage Points 573.3.3 Influential Points and Cook’s Distance 583.4 Example— Estimating Home Prices (continued) 603.5 Summary 634 Transformations and Linearizable Models 674.1 Introduction 674.2 Concepts and Background Material: The Log-Log Model 694.3 Concepts and Background Material: Semilog Models 694.3.1 Logged Response Variable 704.3.2 Logged Predictor Variable 704.4 Example— Predicting Movie Grosses After One Week 714.5 Summary 775 Time Series Data and Autocorrelation 795.1 Introduction 795.2 Concepts and Background Material 815.3 Methodology: Identifying Autocorrelation 835.3.1 The Durbin-Watson Statistic 835.3.2 The Autocorrelation Function (ACF) 845.3.3 Residual Plots and the Runs Test 855.4 Methodology: Addressing Autocorrelation 865.4.1 Detrending and Deseasonalizing 865.4.2 Example— e-Commerce Retail Sales 875.4.3 Lagging and Differencing 935.4.4 Example— Stock Indexes 945.4.5 Generalized Least Squares (GLS): The Cochrane-Orcutt Procedure 995.4.6 Example— Time Intervals Between Old Faithful Geyser Eruptions 1005.5 Summary 104Part III Categorical Predictors6 Analysis of Variance 1096.1 Introduction 1096.2 Concepts and Background Material 1106.2.1 One-Way ANOVA 1106.2.2 Two-Way ANOVA 1116.3 Methodology 1136.3.1 Codings for Categorical Predictors 1136.3.2 Multiple Comparisons 1186.3.3 Levene’s Test and Weighted Least Squares 1206.3.4 Membership in Multiple Groups 1236.4 Example—DVD Sales of Movies 1256.5 Higher-Way ANOVA 1306.6 Summary 1327 Analysis of Covariance 1357.1 Introduction 1357.2 Methodology 1367.2.1 Constant Shift Models 1367.2.2 Varying Slope Models 1377.3 Example —International Grosses of Movies 1377.4 Summary 142Part IV Non-Gaussian Regression Models8 Logistic Regression 1458.1 Introduction 1458.2 Concepts and Background Material 1478.2.1 The Logit Response Function 1488.2.2 Bernoulli and Binomial Random Variables 1498.2.3 Prospective and Retrospective Designs 1498.3 Methodology 1528.3.1 Maximum Likelihood Estimation 1528.3.2 Inference, Model Comparison, and Model Selection 1538.3.3 Goodness-of-Fit 1558.3.4 Measures of Association and Classification Accuracy 1578.3.5 Diagnostics 1598.4 Example— Smoking and Mortality 1598.5 Example— Modeling Bankruptcy 1638.6 Summary 1689 Multinomial Regression 1739.1 Introduction 1739.2 Concepts and Background Material 1749.2.1 Nominal Response Variable 1749.2.2 Ordinal Response Variable 1769.3 Methodology 1789.3.1 Estimation 1789.3.2 Inference, Model Comparisons, and Strength of Fit 1789.3.3 Lack of Fit and Violations of Assumptions 1809.4 Example— City Bond Ratings 1809.5 Summary 18410 Count Regression 18710.1 Introduction 18710.2 Concepts and Background Material 18810.2.1 The Poisson Random Variable 18810.2.2 Generalized Linear Models 18910.3 Methodology 19010.3.1 Estimation and Inference 19010.3.2 Offsets 19110.4 Overdispersion and Negative Binomial Regression 19210.4.1 Quasi-likelihood 19210.4.2 Negative Binomial Regression 19310.5 Example— Unprovoked Shark Attacks in Florida 19410.6 Other Count Regression Models 20110.7 Poisson Regression and Weighted Least Squares 20310.7.1 Example— International Grosses of Movies (continued) 20410.8 Summary 20611 Models for Time-to-Event (Survival) Data 20911.1 Introduction 21011.2 Concepts and Background Material 21111.2.1 The Nature of Survival Data 21111.2.2 Accelerated Failure Time Models 21211.2.3 The Proportional Hazards Model 21411.3 Methodology 21411.3.1 The Kaplan-Meier Estimator and the Log-Rank Test 21411.3.2 Parametric (Likelihood) Estimation 21911.3.3 Semiparametric (Partial Likelihood) Estimation 22111.3.4 The Buckley-James Estimator 22311.4 Example—The Survival of Broadway Shows (continued) 22311.5 Left-Truncated/Right-Censored Data and Time-Varying Covariates 23011.5.1 Left-Truncated/Right-Censored Data 23011.5.2 Example—The Survival of Broadway Shows (continued) 23311.5.3 Time-Varying Covariates 23311.5.4 Example—Female Heads of Government 23511.6 Summary 238Part V Other Regression Models12 Nonlinear Regression 24312.1 Introduction 24312.2 Concepts and Background Material 24412.3 Methodology 24612.3.1 Nonlinear Least Squares Estimation 24612.3.2 Inference for Nonlinear Regression Models 24712.4 Example —Michaelis-Menten Enzyme Kinetics 24812.5 Summary 25213 Models for Longitudinal and Nested Data 25513.1 Introduction 25513.2 Concepts and Background Material 25713.2.1 Nested Data and ANOVA 25713.2.2 Longitudinal Data and Time Series 25813.2.3 Fixed Effects Versus Random Effects 25913.3 Methodology 26013.3.1 The Linear Mixed Effects Model 26013.3.2 The Generalized Linear Mixed Effects Model 26213.3.3 Generalized Estimating Equations 26213.3.4 Nonlinear Mixed Effects Models 26313.4 Example —Tumor Growth in a Cancer Study 26413.5 Example —Unprovoked Shark Attacks in the United States 26913.6 Summary 27514 Regularization Methods and Sparse Models 27714.1 Introduction 27714.2 Concepts and Background Material 27814.2.1 The Bias–Variance Tradeoff 27814.2.2 Large Numbers of Predictors and Sparsity 27914.3 Methodology 28014.3.1 Forward Stepwise Regression 28014.3.2 Ridge Regression 28114.3.3 The Lasso 28114.3.4 Other Regularization Methods 28314.3.5 Choosing the Regularization Parameter(s) 28414.3.6 More Structured Regression Problems 28514.3.7 Cautions About Regularization Methods 28614.4 Example— Human Development Index 28714.5 Summary 289Part VI Nonparametric and Semiparametric Models15 Smoothing and Additive Models 29515.1 Introduction 29615.2 Concepts and Background Material 29615.2.1 The Bias–Variance Tradeoff 29615.2.2 Smoothing and Local Regression 29715.3 Methodology 29815.3.1 Local Polynomial Regression 29815.3.2 Choosing the Bandwidth 29815.3.3 Smoothing Splines 29915.3.4 Multiple Predictors, the Curse of Dimensionality, and Additive Models 30015.4 Example— Prices of German Used Automobiles 30115.5 Local and Penalized Likelihood Regression 30415.5.1 Example— The Bechdel Rule and Hollywood Movies 30515.6 Using Smoothing to Identify Interactions 30715.6.1 Example— Estimating Home Prices (continued) 30815.7 Summary 31016 Tree-Based Models 31316.1 Introduction 31416.2 Concepts and Background Material 31416.2.1 Recursive Partitioning 31416.2.2 Types of Trees 31716.3 Methodology 31816.3.1 CART 31816.3.2 Conditional Inference Trees 31916.3.3 Ensemble Methods 32016.4 Examples 32116.4.1 Estimating Home Prices (continued) 32116.4.2 Example—Courtesy in Airplane Travel 32216.5 Trees for Other Types of Data 32716.5.1 Trees for Nested and Longitudinal Data 32716.5.2 Survival Trees 32816.6 Summary 332Bibliography 337Index 343