Discovering Knowledge in Data

Daniel T. Larose earned his PhD in Statistics at the University of Connecticut. He is Professor of Mathematical Sciences and Director of the Data Mining programs at Central Connecticut State University.  His consulting clients have included Microsoft, Forbes Magazine, the CIT Group, KPMG International, Computer Associates, and Deloitte, Inc. This is Larose’s fourth book for Wiley.Chantal D. Larose is an Assistant Professor of Statistics & Data Science at Eastern Connecticut State University (ECSU).  She has co-authored three books on data science and predictive analytics.  She helped develop data science programs at ECSU and at SUNY New Paltz.  She received her PhD in Statistics from the University of Connecticut, Storrs in 2015 (dissertation title: Model-based Clustering of Incomplete Data).

• Preface xiChapter 1 An Introduction to Data Mining 11.1 What is Data Mining? 11.2 Wanted: Data Miners 21.3 The Need for Human Direction of Data Mining 31.4 The Cross-Industry Standard Practice for Data Mining 41.4.1 Crisp-DM: The Six Phases 51.5 Fallacies of Data Mining 61.6 What Tasks Can Data Mining Accomplish? 81.6.1 Description 81.6.2 Estimation 81.6.3 Prediction 101.6.4 Classification 101.6.5 Clustering 121.6.6 Association 14References 14Exercises 15Chapter 2 Data Preprocessing 162.1 Why do We Need to Preprocess the Data? 172.2 Data Cleaning 172.3 Handling Missing Data 192.4 Identifying Misclassifications 222.5 Graphical Methods for Identifying Outliers 222.6 Measures of Center and Spread 232.7 Data Transformation 262.8 Min-Max Normalization 262.9 Z-Score Standardization 272.10 Decimal Scaling 282.11 Transformations to Achieve Normality 282.12 Numerical Methods for Identifying Outliers 352.13 Flag Variables 362.14 Transforming Categorical Variables into Numerical Variables 372.15 Binning Numerical Variables 382.16 Reclassifying Categorical Variables 392.17 Adding an Index Field 392.18 Removing Variables that are Not Useful 392.19 Variables that Should Probably Not Be Removed 402.20 Removal of Duplicate Records 412.21 A Word About ID Fields 41The R Zone 42References 48Exercises 48Hands-On Analysis 50Chapter 3 Exploratory Data Analysis 513.1 Hypothesis Testing Versus Exploratory Data Analysis 513.2 Getting to Know the Data Set 523.3 Exploring Categorical Variables 553.4 Exploring Numeric Variables 623.5 Exploring Multivariate Relationships 693.6 Selecting Interesting Subsets of the Data for Further Investigation 713.7 Using EDA to Uncover Anomalous Fields 713.8 Binning Based on Predictive Value 723.9 Deriving New Variables: Flag Variables 743.10 Deriving New Variables: Numerical Variables 773.11 Using EDA to Investigate Correlated Predictor Variables 773.12 Summary 80The R Zone 82Reference 88Exercises 88Hands-On Analysis 89Chapter 4 Univariate Statistical Analysis 914.1 Data Mining Tasks in Discovering Knowledge in Data 914.2 Statistical Approaches to Estimation and Prediction 924.3 Statistical Inference 934.4 How Confident are We in Our Estimates? 944.5 Confidence Interval Estimation of the Mean 954.6 How to Reduce the Margin of Error 974.7 Confidence Interval Estimation of the Proportion 984.8 Hypothesis Testing for the Mean 994.9 Assessing the Strength of Evidence Against the Null Hypothesis 1014.10 Using Confidence Intervals to Perform Hypothesis Tests 1024.11 Hypothesis Testing for the Proportion 104The R Zone 105Reference 106Exercises 106Chapter 5 Multivariate Statistics 1095.1 Two-Sample t-Test for Difference in Means 1105.2 Two-Sample Z-Test for Difference in Proportions 1115.3 Test for Homogeneity of Proportions 1125.4 Chi-Square Test for Goodness of Fit of Multinomial Data 1145.5 Analysis of Variance 1155.6 Regression Analysis 1185.7 Hypothesis Testing in Regression 1225.8 Measuring the Quality of a Regression Model 1235.9 Dangers of Extrapolation 1235.10 Confidence Intervals for the Mean Value of y Given x 1255.11 Prediction Intervals for a Randomly Chosen Value of y Given x 1255.12 Multiple Regression 1265.13 Verifying Model Assumptions 127The R Zone 131Reference 135Exercises 135Hands-On Analysis 136Chapter 6 Preparing to Model the Data 1386.1 Supervised Versus Unsupervised Methods 1386.2 Statistical Methodology and Data Mining Methodology 1396.3 Cross-Validation 1396.4 Overfitting 1416.5 BIAS–Variance Trade-Off 1426.6 Balancing the Training Data Set 1446.7 Establishing Baseline Performance 145The R Zone 146Reference 147Exercises 147Chapter 7 K-Nearest Neighbor Algorithm 1497.1 Classification Task 1497.2 k-Nearest Neighbor Algorithm 1507.3 Distance Function 1537.4 Combination Function 1567.4.1 Simple Unweighted Voting 1567.4.2 Weighted Voting 1567.5 Quantifying Attribute Relevance: Stretching the Axes 1587.6 Database Considerations 1587.7 k-Nearest Neighbor Algorithm for Estimation and Prediction 1597.8 Choosing k 1607.9 Application of k-Nearest Neighbor Algorithm Using IBM/SPSS Modeler 160The R Zone 162Exercises 163Hands-On Analysis 164Chapter 8 Decision Trees 1658.1 What is a Decision Tree? 1658.2 Requirements for Using Decision Trees 1678.3 Classification and Regression Trees 1688.4 C4.5 Algorithm 1748.5 Decision Rules 1798.6 Comparison of the C5.0 and Cart Algorithms Applied to Real Data 180The R Zone 183References 184Exercises 185Hands-On Analysis 185Chapter 9 Neural Networks 1879.1 Input and Output Encoding 1889.2 Neural Networks for Estimation and Prediction 1909.3 Simple Example of a Neural Network 1919.4 Sigmoid Activation Function 1939.5 Back-Propagation 1949.5.1 Gradient Descent Method 1949.5.2 Back-Propagation Rules 1959.5.3 Example of Back-Propagation 1969.6 Termination Criteria 1989.7 Learning Rate 1989.8 Momentum Term 1999.9 Sensitivity Analysis 2019.10 Application of Neural Network Modeling 202The R Zone 204References 207Exercises 207Hands-On Analysis 207Chapter 10 Hierarchical and K-Means Clustering 20910.1 The Clustering Task 20910.2 Hierarchical Clustering Methods 21210.3 Single-Linkage Clustering 21310.4 Complete-Linkage Clustering 21410.5 k-Means Clustering 21510.6 Example of k-Means Clustering at Work 21610.7 Behavior of MSB, MSE, and PSEUDO-F as the k-Means Algorithm Proceeds 21910.8 Application of k-Means Clustering Using SAS Enterprise Miner 22010.9 Using Cluster Membership to Predict Churn 223The R Zone 224References 226Exercises 226Hands-On Analysis 226Chapter 11 Kohonen Networks 22811.1 Self-Organizing Maps 22811.2 Kohonen Networks 23011.2.1 Kohonen Networks Algorithm 23111.3 Example of a Kohonen Network Study 23111.4 Cluster Validity 23511.5 Application of Clustering Using Kohonen Networks 23511.6 Interpreting the Clusters 23711.6.1 Cluster Profiles 24011.7 Using Cluster Membership as Input to Downstream Data Mining Models 242The R Zone 243References 245Exercises 245Hands-On Analysis 245Chapter 12 Association Rules 24712.1 Affinity Analysis and Market Basket Analysis 24712.1.1 Data Representation for Market Basket Analysis 24812.2 Support, Confidence, Frequent Itemsets, and the a Priori Property 24912.3 How Does the a Priori Algorithm Work? 25112.3.1 Generating Frequent Itemsets 25112.3.2 Generating Association Rules 25312.4 Extension from Flag Data to General Categorical Data 25512.5 Information-Theoretic Approach: Generalized Rule Induction Method 25612.5.1 J-Measure 25712.6 Association Rules are Easy to do Badly 25812.7 How Can We Measure the Usefulness of Association Rules? 25912.8 Do Association Rules Represent Supervised or Unsupervised Learning? 26012.9 Local Patterns Versus Global Models 261The R Zone 262References 263Exercises 263Hands-On Analysis 264Chapter 13 Imputation of Missing Data 26613.1 Need for Imputation of Missing Data 26613.2 Imputation of Missing Data: Continuous Variables 26713.3 Standard Error of the Imputation 27013.4 Imputation of Missing Data: Categorical Variables 27113.5 Handling Patterns in Missingness 272The R Zone 273Reference 276Exercises 276Hands-On Analysis 276Chapter 14 Model Evaluation Techniques 27714.1 Model Evaluation Techniques for the Description Task 27814.2 Model Evaluation Techniques for the Estimation and Prediction Tasks 27814.3 Model Evaluation Techniques for the Classification Task 28014.4 Error Rate, False Positives, and False Negatives 28014.5 Sensitivity and Specificity 28314.6 Misclassification Cost Adjustment to Reflect Real-World Concerns 28414.7 Decision Cost/Benefit Analysis 28514.8 Lift Charts and Gains Charts 28614.9 Interweaving Model Evaluation with Model Building 28914.10 Confluence of Results: Applying a Suite of Models 290The R Zone 291Reference 291Exercises 291Hands-On Analysis 291Appendix: Data Summarization and Visualization 294Index 309