Statistical Data Analytics

WALTER W. PIEGORSCH University of Arizona, USA

• Preface xiii Part I Background: Introductory Statistical Analytics 11 Data analytics and data mining 31.1 Knowledge discovery: finding structure in data 31.2 Data quality versus data quantity 51.3 Statistical modeling versus statistical description 72 Basic probability and statistical distributions 102.1 Concepts in probability 102.1.1 Probability rules 112.1.2 Random variables and probability functions 122.1.3 Means, variances, and expected values 172.1.4 Median, quartiles, and quantiles 182.1.5 Bivariate expected values, covariance, and correlation 202.2 Multiple random variables∗ 212.3 Univariate families of distributions 232.3.1 Binomial distribution 232.3.2 Poisson distribution 262.3.3 Geometric distribution 272.3.4 Negative binomial distribution 272.3.5 Discrete uniform distribution 282.3.6 Continuous uniform distribution 292.3.7 Exponential distribution 292.3.8 Gamma and chi-square distributions 302.3.9 Normal (Gaussian) distribution 322.3.10 Distributions derived from normal 372.3.11 The exponential family 413 Data manipulation 493.1 Random sampling 493.2 Data types 513.3 Data summarization 523.3.1 Means, medians, and central tendency 523.3.2 Summarizing variation 563.3.3 Summarizing (bivariate) correlation 593.4 Data diagnostics and data transformation 603.4.1 Outlier analysis 603.4.2 Entropy∗ 623.4.3 Data transformation 643.5 Simple smoothing techniques 653.5.1 Binning 663.5.2 Moving averages∗ 673.5.3 Exponential smoothing∗ 694 Data visualization and statistical graphics 764.1 Univariate visualization 774.1.1 Strip charts and dot plots 774.1.2 Boxplots 794.1.3 Stem-and-leaf plots 814.1.4 Histograms and density estimators 834.1.5 Quantile plots 874.2 Bivariate and multivariate visualization 894.2.1 Pie charts and bar charts 904.2.2 Multiple boxplots and QQ plots 954.2.3 Scatterplots and bubble plots 984.2.4 Heatmaps 1024.2.5 Time series plots∗ 1055 Statistical inference 1155.1 Parameters and likelihood 1155.2 Point estimation 1175.2.1 Bias 1185.2.2 The method of moments 1185.2.3 Least squares/weighted least squares 1195.2.4 Maximum likelihood∗ 1205.3 Interval estimation 1235.3.1 Confidence intervals 1235.3.2 Single-sample intervals for normal (Gaussian) parameters 1245.3.3 Two-sample intervals for normal (Gaussian) parameters 1285.3.4 Wald intervals and likelihood intervals∗ 1315.3.5 Delta method intervals∗ 1355.3.6 Bootstrap intervals∗ 1375.4 Testing hypotheses 1385.4.1 Single-sample tests for normal (Gaussian) parameters 1405.4.2 Two-sample tests for normal (Gaussian) parameters 1425.4.3 Walds tests, likelihood ratio tests, and ‘exact’ tests∗ 1455.5 Multiple inferences∗ 1485.5.1 Bonferroni multiplicity adjustment 1495.5.2 False discovery rate 151Part II Statistical Learning and Data Analytics 1616 Techniques for supervised learning: simple linear regression 1636.1 What is “supervised learning?” 1636.2 Simple linear regression 1646.2.1 The simple linear model 1646.2.2 Multiple inferences and simultaneous confidence bands 1716.3 Regression diagnostics 1756.4 Weighted least squares (WLS) regression 1846.5 Correlation analysis 1876.5.1 The correlation coefficient 1876.5.2 Rank correlation 1907 Techniques for supervised learning: multiple linear regression 1987.1 Multiple linear regression 1987.1.1 Matrix formulation 1997.1.2 Weighted least squares for the MLR model 2007.1.3 Inferences under the MLR model 2017.1.4 Multicollinearity 2087.2 Polynomial regression 2107.3 Feature selection 2117.3.1 R2p plots 2127.3.2 Information criteria: AIC and BIC 2157.3.3 Automated variable selection 2167.4 Alternative regression methods∗ 2237.4.1 Loess 2247.4.2 Regularization: ridge regression 2307.4.3 Regularization and variable selection: the Lasso 2387.5 Qualitative predictors: ANOVA models 2428 Supervised learning: generalized linear models 2588.1 Extending the linear regression model 2588.1.1 Nonnormal data and the exponential family 2588.1.2 Link functions 2598.2 Technical details for GLiMs∗ 2598.2.1 Estimation 2608.2.2 The deviance function 2618.2.3 Residuals 2628.2.4 Inference and model assessment 2648.3 Selected forms of GLiMs 2658.3.1 Logistic regression and binary-data GLiMs 2658.3.2 Trend testing with proportion data 2718.3.3 Contingency tables and log-linear models 2738.3.4 Gamma regression models 2819 Supervised learning: classification 2919.1 Binary classification via logistic regression 2929.1.1 Logistic discriminants 2929.1.2 Discriminant rule accuracy 2969.1.3 ROC curves 2979.2 Linear discriminant analysis (LDA) 2979.2.1 Linear discriminant functions 2979.2.2 Bayes discriminant/classification rules 3029.2.3 Bayesian classification with normal data 3039.2.4 Naïve Bayes classifiers 3089.3 k-Nearest neighbor classifiers 3089.4 Tree-based methods 3129.4.1 Classification trees 3129.4.2 Pruning 3149.4.3 Boosting 3219.4.4 Regression trees 3219.5 Support vector machines∗ 3229.5.1 Separable data 3229.5.2 Nonseparable data 3259.5.3 Kernel transformations 32610 Techniques for unsupervised learning: dimension reduction 34110.1 Unsupervised versus supervised learning 34110.2 Principal component analysis 34210.2.1 Principal components 34210.2.2 Implementing a PCA 34410.3 Exploratory factor analysis 35110.3.1 The factor analytic model 35110.3.2 Principal factor estimation 35310.3.3 Maximum likelihood estimation 35410.3.4 Selecting the number of factors 35510.3.5 Factor rotation 35610.3.6 Implementing an EFA 35710.4 Canonical correlation analysis∗ 36111 Techniques for unsupervised learning: clustering and association 37311.1 Cluster analysis 37311.1.1 Hierarchical clustering 37611.1.2 Partitioned clustering 38411.2 Association rules/market basket analysis 39511.2.1 Association rules for binary observations 39611.2.2 Measures of rule quality 39711.2.3 The Apriori algorithm 39811.2.4 Statistical measures of association quality 402A Matrix manipulation 411A.1 Vectors and matrices 411A.2 Matrix algebra 412A.3 Matrix inversion 414A.4 Quadratic forms 415A.5 Eigenvalues and eigenvectors 415A.6 Matrix factorizations 416A.6.1 QR decomposition 417A.6.2 Spectral decomposition 417A.6.3 Matrix square root 417A.6.4 Singular value decomposition 418A.7 Statistics via matrix operations 419B Brief introduction to R 421B.1 Data entry and manipulation 422B.2 A turbo-charged calculator 426B.3 R functions 427B.3.1 Inbuilt R functions 427B.3.2 Flow control 429B.3.3 User-defined functions 429B.4 R packages 430References 432Index 453