Applied Predictive Analytics

DEAN ABBOTT is President of Abbott Analytics, Inc. (San Diego). He is an internationally recognized data mining and predictive analytics expert with over two decades experience in fraud detection, risk modeling, text mining, personality assessment, planned giving, toxicology, and other applications. He is also Chief Scientist of SmarterRemarketer, a company focusing on behaviorally- and data-driven marketing and web analytics.

"This book provides an excellent background to predictive analytics" (BCS, December 2014)

• Introduction xxiChapter 1 Overview of Predictive Analytics 1What Is Analytics? 3What Is Predictive Analytics? 3Supervised vs. Unsupervised Learning 5Parametric vs. Non-Parametric Models 6Business Intelligence 6Predictive Analytics vs. Business Intelligence 8Do Predictive Models Just State the Obvious? 9Similarities between Business Intelligence and Predictive Analytics 9Predictive Analytics vs. Statistics 10Statistics and Analytics 11Predictive Analytics and Statistics Contrasted 12Predictive Analytics vs. Data Mining 13Who Uses Predictive Analytics? 13Challenges in Using Predictive Analytics 14Obstacles in Management 14Obstacles with Data 14Obstacles with Modeling 15Obstacles in Deployment 16What Educational Background Is Needed to Become a Predictive Modeler? 16Chapter 2 Setting Up the Problem 19Predictive Analytics Processing Steps: CRISP-DM 19Business Understanding 21The Three-Legged Stool 22Business Objectives 23Defining Data for Predictive Modeling 25Defining the Columns as Measures 26Defining the Unit of Analysis 27Which Unit of Analysis? 28Defining the Target Variable 29Temporal Considerations for Target Variable 31Defining Measures of Success for Predictive Models 32Success Criteria for Classification 32Success Criteria for Estimation 33Other Customized Success Criteria 33Doing Predictive Modeling Out of Order 34Building Models First 34Early Model Deployment 35Case Study: Recovering Lapsed Donors 35Overview 36Business Objectives 36Data for the Competition 36The Target Variables 36Modeling Objectives 37Model Selection and Evaluation Criteria 38Model Deployment 39Case Study: Fraud Detection 39Overview 39Business Objectives 39Data for the Project 40The Target Variables 40Modeling Objectives 41Model Selection and Evaluation Criteria 41Model Deployment 41Summary 42Chapter 3 Data Understanding 43What the Data Looks Like 44Single Variable Summaries 44Mean 45Standard Deviation 45The Normal Distribution 45Uniform Distribution 46Applying Simple Statistics in Data Understanding 47Skewness 49Kurtosis 51Rank-Ordered Statistics 52Categorical Variable Assessment 55Data Visualization in One Dimension 58Histograms 59Multiple Variable Summaries 64Hidden Value in Variable Interactions: Simpson’s Paradox 64The Combinatorial Explosion of Interactions 65Correlations 66Spurious Correlations 66Back to Correlations 67Crosstabs 68Data Visualization, Two or Higher Dimensions 69Scatterplots 69Anscombe’s Quartet 71Scatterplot Matrices 75Overlaying the Target Variable in Summary 76Scatterplots in More Than Two Dimensions 78The Value of Statistical Significance 80Pulling It All Together into a Data Audit 81Summary 82Chapter 4 Data Preparation 83Variable Cleaning 84Incorrect Values 84Consistency in Data Formats 85Outliers 85Multidimensional Outliers 89Missing Values 90Fixing Missing Data 91Feature Creation 98Simple Variable Transformations 98Fixing Skew 99Binning Continuous Variables 103Numeric Variable Scaling 104Nominal Variable Transformation 107Ordinal Variable Transformations 108Date and Time Variable Features 109ZIP Code Features 110Which Version of a Variable Is Best? 110Multidimensional Features 112Variable Selection Prior to Modeling 117Sampling 123Example: Why Normalization Matters for K-Means Clustering 139Summary 143Chapter 5 Itemsets and Association Rules 145Terminology 146Condition 147Left-Hand-Side, Antecedent(s) 148Right-Hand-Side, Consequent, Output, Conclusion 148Rule (Item Set) 148Support 149Antecedent Support 149Confidence, Accuracy 150Lift 150Parameter Settings 151How the Data Is Organized 151Standard Predictive Modeling Data Format 151Transactional Format 152Measures of Interesting Rules 154Deploying Association Rules 156Variable Selection 157Interaction Variable Creation 157Problems with Association Rules 158Redundant Rules 158Too Many Rules 158Too Few Rules 159Building Classification Rules from Association Rules 159Summary 161Chapter 6 Descriptive Modeling 163Data Preparation Issues with Descriptive Modeling 164Principal Component Analysis 165The PCA Algorithm 165Applying PCA to New Data 169PCA for Data Interpretation 171Additional Considerations before Using PCA 172The Effect of Variable Magnitude on PCA Models 174Clustering Algorithms 177The K-Means Algorithm 178Data Preparation for K-Means 183Selecting the Number of Clusters 185The Kohonen SOM Algorithm 192Visualizing Kohonen Maps 194Similarities with K-Means 196Summary 197Chapter 7 Interpreting Descriptive Models 199Standard Cluster Model Interpretation 199Problems with Interpretation Methods 202Identifying Key Variables in Forming Cluster Models 203Cluster Prototypes 209Cluster Outliers 210Summary 212Chapter 8 Predictive Modeling 213Decision Trees 214The Decision Tree Landscape 215Building Decision Trees 218Decision Tree Splitting Metrics 221Decision Tree Knobs and Options 222Reweighting Records: Priors 224Reweighting Records: Misclassification Costs 224Other Practical Considerations for Decision Trees 229Logistic Regression 230Interpreting Logistic Regression Models 233Other Practical Considerations for Logistic Regression 235Neural Networks 240Building Blocks: The Neuron 242Neural Network Training 244The Flexibility of Neural Networks 247Neural Network Settings 249Neural Network Pruning 251Interpreting Neural Networks 252Neural Network Decision Boundaries 253Other Practical Considerations for Neural Networks 253K-Nearest Neighbor 254The k-NN Learning Algorithm 254Distance Metrics for k-NN 258Other Practical Considerations for k-NN 259Naïve Bayes 264Bayes’ Theorem 264The Naïve Bayes Classifier 268Interpreting Naïve Bayes Classifiers 268Other Practical Considerations for Naïve Bayes 269Regression Models 270Linear Regression 271Linear Regression Assumptions 274Variable Selection in Linear Regression 276Interpreting Linear Regression Models 278Using Linear Regression for Classification 279Other Regression Algorithms 280Summary 281Chapter 9 Assessing Predictive Models 283Batch Approach to Model Assessment 284Percent Correct Classification 284Rank-Ordered Approach to Model Assessment 293Assessing Regression Models 301Summary 304Chapter 10 Model Ensembles 307Motivation for Ensembles 307The Wisdom of Crowds 308Bias Variance Tradeoff 309Bagging 311Boosting 316Improvements to Bagging and Boosting 320Random Forests 320Stochastic Gradient Boosting 321Heterogeneous Ensembles 321Model Ensembles and Occam’s Razor 323Interpreting Model Ensembles 323Summary 326Chapter 11 Text Mining 327Motivation for Text Mining 328A Predictive Modeling Approach to Text Mining 329Structured vs. Unstructured Data 329Why Text Mining Is Hard 330Text Mining Applications 332Data Sources for Text Mining 333Data Preparation Steps 333POS Tagging 333Tokens 336Stop Word and Punctuation Filters 336Character Length and Number Filters 337Stemming 337Dictionaries 338The Sentiment Polarity Movie Data Set 339Text Mining Features 340Term Frequency 341Inverse Document Frequency 344Tf-idf 344Cosine Similarity 346Multi-Word Features: N-Grams 346Reducing Keyword Features 347Grouping Terms 347Modeling with Text Mining Features 347Regular Expressions 349Uses of Regular Expressions in Text Mining 351Summary 352Chapter 12 Model Deployment 353General Deployment Considerations 354Deployment Steps 355Summary 375Chapter 13 Case Studies 377Survey Analysis Case Study: Overview 377Business Understanding: Defining the Problem 378Data Understanding 380Data Preparation 381Modeling 385Deployment: “What-If” Analysis 391Revisit Models 392Deployment 401Summary and Conclusions 401Help Desk Case Study 402Data Understanding: Defining the Data 403Data Preparation 403Modeling 405Revisit Business Understanding 407Deployment 409Summary and Conclusions 411Index 413