João Mendes Moreira, PhD, is an assistant professor in the Faculty of Engineering at the University of Porto, Porto, Portugal and is also a researcher in LIAAD-INESC TEC, Porto, Portugal. André de Carvalho, PhD, is a full professor in the Institute of Mathematics and Computer Science at the University of São Paulo, Brazil. Tomáš Horváth, PhD, is an assistant professor at the Faculty of Informatics of the Eötvös Loránd University in Budapest, Hungary, and is also associated with the Faculty of Science at the Pavol Jozef Šafárik University in Košice, Slovakia.

• Preface xiiiAcknowledgments xvPresentational Conventions xviiAbout the Companion Website xixPart I Introductory Background 11 What Can We Do With Data? 31.1 Big Data and Data Science 41.2 Big Data Architectures 51.3 Small Data 61.4 What is Data? 71.5 A Short Taxonomy of Data Analytics 91.6 Examples of Data Use 101.6.1 Breast Cancer in Wisconsin 111.6.2 Polish Company Insolvency Data 111.7 A Project on Data Analytics 121.7.1 A Little History on Methodologies for Data Analytics 121.7.2 The KDD Process 141.7.3 The CRISP-DM Methodology 151.8 How this Book is Organized 161.9 Who Should Read this Book 18Part II Getting Insights from Data 192 Descriptive Statistics 212.1 Scale Types 222.2 Descriptive Univariate Analysis 252.2.1 Univariate Frequencies 252.2.2 Univariate Data Visualization 272.2.3 Univariate Statistics 322.2.4 Common Univariate Probability Distributions 382.3 Descriptive Bivariate Analysis 402.3.1 Two Quantitative Attributes 412.3.2 Two Qualitative Attributes, at Least one of them Nominal 452.3.3 Two Ordinal Attributes 462.4 Final Remarks 472.5 Exercises 473 Descriptive Multivariate Analysis 493.1 Multivariate Frequencies 493.2 Multivariate Data Visualization 503.3 Multivariate Statistics 593.3.1 Location Multivariate Statistics 593.3.2 Dispersion Multivariate Statistics 603.4 Infographics and Word Clouds 663.4.1 Infographics 663.4.2 Word Clouds 673.5 Final Remarks 673.6 Exercises 684 Data Quality and Preprocessing 714.1 Data Quality 714.1.1 Missing Values 724.1.2 Redundant Data 744.1.3 Inconsistent Data 754.1.4 Noisy Data 764.1.5 Outliers 774.2 Converting to a Different Scale Type 774.2.1 Converting Nominal to Relative 784.2.2 Converting Ordinal to Relative or Absolute 814.2.3 Converting Relative or Absolute to Ordinal or Nominal 824.3 Converting to a Different Scale 834.4 Data Transformation 854.5 Dimensionality Reduction 864.5.1 Attribute Aggregation 884.5.1.1 Principal Component Analysis 884.5.1.2 Independent Component Analysis 914.5.1.3 Multidimensional Scaling 914.5.2 Attribute Selection 924.5.2.1 Filters 924.5.2.2 Wrappers 934.5.2.3 Embedded 944.5.2.4 Search Strategies 954.6 Final Remarks 964.7 Exercises 965 Clustering 995.1 Distance Measures 1005.1.1 Differences between Values of Common Attribute Types 1015.1.2 Distance Measures for Objects with Quantitative Attributes 1035.1.3 Distance Measures for Non-conventional Attributes 1045.2 Clustering Validation 1075.3 Clustering Techniques 1085.3.1 K-means 1105.3.1.1 Centroids and Distance Measures 1105.3.1.2 How K-means Works 1115.3.2 DBSCAN 1155.3.3 Agglomerative Hierarchical Clustering Technique 1175.3.3.1 Linkage Criterion 1195.3.3.2 Dendrograms 1205.4 Final Remarks 1225.5 Exercises 1236 Frequent Pattern Mining 1256.1 Frequent Itemsets 1276.1.1 Setting the min_sup Threshold 1286.1.2 Apriori – a Join-based Method 1316.1.3 Eclat 1336.1.4 FP-Growth 1346.1.5 Maximal and Closed Frequent Itemsets 1386.2 Association Rules 1396.3 Behind Support and Confidence 1426.3.1 Cross-support Patterns 1436.3.2 Lift 1446.3.3 Simpson’s Paradox 1456.4 Other Types of Pattern 1476.4.1 Sequential patterns 1476.4.2 Frequent Sequence Mining 1486.4.3 Closed and Maximal Sequences 1486.5 Final Remarks 1496.6 Exercises 1497 Cheat Sheet and Project on Descriptive Analytics 1517.1 Cheat Sheet of Descriptive Analytics 1517.1.1 On Data Summarization 1517.1.2 On Clustering 1517.1.3 On Frequent Pattern Mining 1537.2 Project on Descriptive Analytics 1547.2.1 Business Understanding 1547.2.2 Data Understanding 1557.2.3 Data Preparation 1557.2.4 Modeling 1577.2.5 Evaluation 1587.2.6 Deployment 158Part III Predicting the Unknown 1598 Regression 1618.1 Predictive Performance Estimation 1648.1.1 Generalization 1648.1.2 Model Validation 1658.1.3 Predictive Performance Measures for Regression 1698.2 Finding the Parameters of the Model 1718.2.1 Linear Regression 1718.2.1.1 Empirical Error 1738.2.2 The Bias-variance Trade-off 1758.2.3 Shrinkage Methods 1778.2.3.1 Ridge Regression 1798.2.3.2 Lasso Regression 1808.2.4 Methods that use Linear Combinations of Attributes 1818.2.4.1 Principal Components Regression 1818.2.4.2 Partial Least Squares Regression 1828.3 Technique and Model Selection 1828.4 Final Remarks 1838.5 Exercises 1849 Classification 1879.1 Binary Classification 1889.2 Predictive Performance Measures for Classification 1929.3 Distance-based Learning Algorithms 1999.3.1 K-nearest Neighbor Algorithms 1999.3.2 Case-based Reasoning 2029.4 Probabilistic Classification Algorithms 2039.4.1 Logistic Regression Algorithm 2059.4.2 Naive Bayes Algorithm 2079.5 Final Remarks 2089.6 Exercises 21010 Additional Predictive Methods 21110.1 Search-based Algorithms 21110.1.1 Decision Tree Induction Algorithms 21210.1.2 Decision Trees for Regression 21710.1.2.1 Model Trees 21810.1.2.2 Multivariate Adaptive Regression Splines 21910.2 Optimization-based Algorithms 22110.2.1 Artificial Neural Networks 22210.2.1.1 Backpropagation 22410.2.1.2 Deep Networks and Deep Learning Algorithms 23010.2.2 Support Vector Machines 23310.2.2.1 SVM for Regression 23710.3 Final Remarks 23810.4 Exercises 23911 Advanced Predictive Topics 24111.1 Ensemble Learning 24111.1.1 Bagging 24311.1.2 Random Forests 24411.1.3 AdaBoost 24511.2 Algorithm Bias 24611.3 Non-binary Classification Tasks 24811.3.1 One-class Classification 24811.3.2 Multi-class Classification 24911.3.3 Ranking Classification 25011.3.4 Multi-label Classification 25111.3.5 Hierarchical Classification 25211.4 Advanced Data Preparation Techniques for Prediction 25311.4.1 Imbalanced Data Classification 25311.4.2 For Incomplete Target Labeling 25411.4.2.1 Semi-supervised Learning 25411.4.2.2 Active Learning 25511.5 Description and Prediction with Supervised Interpretable Techniques 25511.6 Exercises 25612 Cheat Sheet and Project on Predictive Analytics 25912.1 Cheat Sheet on Predictive Analytics 25912.2 Project on Predictive Analytics 25912.2.1 Business Understanding 26012.2.2 Data Understanding 26012.2.3 Data Preparation 26512.2.4 Modeling 26512.2.5 Evaluation 26512.2.6 Deployment 266Part IV Popular Data Analytics Applications 26713 Applications for Text, Web and Social Media 26913.1 Working with Texts 26913.1.1 Data Acquisition 27113.1.2 Feature Extraction 27113.1.2.1 Tokenization 27213.1.2.2 Stemming 27213.1.2.3 Conversion to Structured Data 27513.1.2.4 Is the Bag of Words Enough? 27613.1.3 Remaining Phases 27713.1.4 Trends 27713.1.4.1 Sentiment Analysis 27813.1.4.2 Web Mining 27813.2 Recommender Systems 27813.2.1 Feedback 27913.2.2 Recommendation Tasks 28013.2.3 Recommendation Techniques 28113.2.3.1 Knowledge-based Techniques 28113.2.3.2 Content-based Techniques 28213.2.3.3 Collaborative Filtering Techniques 28213.2.4 Final Remarks 28913.3 Social Network Analysis 29113.3.1 Representing Social Networks 29113.3.2 Basic Properties of Nodes 29413.3.2.1 Degree 29413.3.2.2 Distance 29413.3.2.3 Closeness 29513.3.2.4 Betweenness 29613.3.2.5 Clustering Coefficient 29713.3.3 Basic and Structural Properties of Networks 29713.3.3.1 Diameter 29713.3.3.2 Centralization 29713.3.3.3 Cliques 29913.3.3.4 Clustering Coefficient 29913.3.3.5 Modularity 29913.3.4 Trends and Final Remarks 29913.4 Exercises 300Apendix A: Comprehensive Description of the CRISP-DM Methodology 303References 311Index 315

João Mendes Moreira, PhD, is an assistant professor in the Faculty of Engineering at the University of Porto, Porto, Portugal and is also a researcher in LIAAD-INESC TEC, Porto, Portugal. André de Carvalho, PhD, is a full professor in the Institute of Mathematics and Computer Science at the University of São Paulo, Brazil. Tomáš Horváth, PhD, is an assistant professor at the Faculty of Informatics of the Eötvös Loránd University in Budapest, Hungary, and is also associated with the Faculty of Science at the Pavol Jozef Šafárik University in Košice, Slovakia.

• Preface xiiiAcknowledgments xvPresentational Conventions xviiAbout the Companion Website xixPart I Introductory Background 11 What Can We Do With Data? 31.1 Big Data and Data Science 41.2 Big Data Architectures 51.3 Small Data 61.4 What is Data? 71.5 A Short Taxonomy of Data Analytics 91.6 Examples of Data Use 101.6.1 Breast Cancer in Wisconsin 111.6.2 Polish Company Insolvency Data 111.7 A Project on Data Analytics 121.7.1 A Little History on Methodologies for Data Analytics 121.7.2 The KDD Process 141.7.3 The CRISP-DM Methodology 151.8 How this Book is Organized 161.9 Who Should Read this Book 18Part II Getting Insights from Data 192 Descriptive Statistics 212.1 Scale Types 222.2 Descriptive Univariate Analysis 252.2.1 Univariate Frequencies 252.2.2 Univariate Data Visualization 272.2.3 Univariate Statistics 322.2.4 Common Univariate Probability Distributions 382.3 Descriptive Bivariate Analysis 402.3.1 Two Quantitative Attributes 412.3.2 Two Qualitative Attributes, at Least one of them Nominal 452.3.3 Two Ordinal Attributes 462.4 Final Remarks 472.5 Exercises 473 Descriptive Multivariate Analysis 493.1 Multivariate Frequencies 493.2 Multivariate Data Visualization 503.3 Multivariate Statistics 593.3.1 Location Multivariate Statistics 593.3.2 Dispersion Multivariate Statistics 603.4 Infographics and Word Clouds 663.4.1 Infographics 663.4.2 Word Clouds 673.5 Final Remarks 673.6 Exercises 684 Data Quality and Preprocessing 714.1 Data Quality 714.1.1 Missing Values 724.1.2 Redundant Data 744.1.3 Inconsistent Data 754.1.4 Noisy Data 764.1.5 Outliers 774.2 Converting to a Different Scale Type 774.2.1 Converting Nominal to Relative 784.2.2 Converting Ordinal to Relative or Absolute 814.2.3 Converting Relative or Absolute to Ordinal or Nominal 824.3 Converting to a Different Scale 834.4 Data Transformation 854.5 Dimensionality Reduction 864.5.1 Attribute Aggregation 884.5.1.1 Principal Component Analysis 884.5.1.2 Independent Component Analysis 914.5.1.3 Multidimensional Scaling 914.5.2 Attribute Selection 924.5.2.1 Filters 924.5.2.2 Wrappers 934.5.2.3 Embedded 944.5.2.4 Search Strategies 954.6 Final Remarks 964.7 Exercises 965 Clustering 995.1 Distance Measures 1005.1.1 Differences between Values of Common Attribute Types 1015.1.2 Distance Measures for Objects with Quantitative Attributes 1035.1.3 Distance Measures for Non-conventional Attributes 1045.2 Clustering Validation 1075.3 Clustering Techniques 1085.3.1 K-means 1105.3.1.1 Centroids and Distance Measures 1105.3.1.2 How K-means Works 1115.3.2 DBSCAN 1155.3.3 Agglomerative Hierarchical Clustering Technique 1175.3.3.1 Linkage Criterion 1195.3.3.2 Dendrograms 1205.4 Final Remarks 1225.5 Exercises 1236 Frequent Pattern Mining 1256.1 Frequent Itemsets 1276.1.1 Setting the min_sup Threshold 1286.1.2 Apriori – a Join-based Method 1316.1.3 Eclat 1336.1.4 FP-Growth 1346.1.5 Maximal and Closed Frequent Itemsets 1386.2 Association Rules 1396.3 Behind Support and Confidence 1426.3.1 Cross-support Patterns 1436.3.2 Lift 1446.3.3 Simpson’s Paradox 1456.4 Other Types of Pattern 1476.4.1 Sequential patterns 1476.4.2 Frequent Sequence Mining 1486.4.3 Closed and Maximal Sequences 1486.5 Final Remarks 1496.6 Exercises 1497 Cheat Sheet and Project on Descriptive Analytics 1517.1 Cheat Sheet of Descriptive Analytics 1517.1.1 On Data Summarization 1517.1.2 On Clustering 1517.1.3 On Frequent Pattern Mining 1537.2 Project on Descriptive Analytics 1547.2.1 Business Understanding 1547.2.2 Data Understanding 1557.2.3 Data Preparation 1557.2.4 Modeling 1577.2.5 Evaluation 1587.2.6 Deployment 158Part III Predicting the Unknown 1598 Regression 1618.1 Predictive Performance Estimation 1648.1.1 Generalization 1648.1.2 Model Validation 1658.1.3 Predictive Performance Measures for Regression 1698.2 Finding the Parameters of the Model 1718.2.1 Linear Regression 1718.2.1.1 Empirical Error 1738.2.2 The Bias-variance Trade-off 1758.2.3 Shrinkage Methods 1778.2.3.1 Ridge Regression 1798.2.3.2 Lasso Regression 1808.2.4 Methods that use Linear Combinations of Attributes 1818.2.4.1 Principal Components Regression 1818.2.4.2 Partial Least Squares Regression 1828.3 Technique and Model Selection 1828.4 Final Remarks 1838.5 Exercises 1849 Classification 1879.1 Binary Classification 1889.2 Predictive Performance Measures for Classification 1929.3 Distance-based Learning Algorithms 1999.3.1 K-nearest Neighbor Algorithms 1999.3.2 Case-based Reasoning 2029.4 Probabilistic Classification Algorithms 2039.4.1 Logistic Regression Algorithm 2059.4.2 Naive Bayes Algorithm 2079.5 Final Remarks 2089.6 Exercises 21010 Additional Predictive Methods 21110.1 Search-based Algorithms 21110.1.1 Decision Tree Induction Algorithms 21210.1.2 Decision Trees for Regression 21710.1.2.1 Model Trees 21810.1.2.2 Multivariate Adaptive Regression Splines 21910.2 Optimization-based Algorithms 22110.2.1 Artificial Neural Networks 22210.2.1.1 Backpropagation 22410.2.1.2 Deep Networks and Deep Learning Algorithms 23010.2.2 Support Vector Machines 23310.2.2.1 SVM for Regression 23710.3 Final Remarks 23810.4 Exercises 23911 Advanced Predictive Topics 24111.1 Ensemble Learning 24111.1.1 Bagging 24311.1.2 Random Forests 24411.1.3 AdaBoost 24511.2 Algorithm Bias 24611.3 Non-binary Classification Tasks 24811.3.1 One-class Classification 24811.3.2 Multi-class Classification 24911.3.3 Ranking Classification 25011.3.4 Multi-label Classification 25111.3.5 Hierarchical Classification 25211.4 Advanced Data Preparation Techniques for Prediction 25311.4.1 Imbalanced Data Classification 25311.4.2 For Incomplete Target Labeling 25411.4.2.1 Semi-supervised Learning 25411.4.2.2 Active Learning 25511.5 Description and Prediction with Supervised Interpretable Techniques 25511.6 Exercises 25612 Cheat Sheet and Project on Predictive Analytics 25912.1 Cheat Sheet on Predictive Analytics 25912.2 Project on Predictive Analytics 25912.2.1 Business Understanding 26012.2.2 Data Understanding 26012.2.3 Data Preparation 26512.2.4 Modeling 26512.2.5 Evaluation 26512.2.6 Deployment 266Part IV Popular Data Analytics Applications 26713 Applications for Text, Web and Social Media 26913.1 Working with Texts 26913.1.1 Data Acquisition 27113.1.2 Feature Extraction 27113.1.2.1 Tokenization 27213.1.2.2 Stemming 27213.1.2.3 Conversion to Structured Data 27513.1.2.4 Is the Bag of Words Enough? 27613.1.3 Remaining Phases 27713.1.4 Trends 27713.1.4.1 Sentiment Analysis 27813.1.4.2 Web Mining 27813.2 Recommender Systems 27813.2.1 Feedback 27913.2.2 Recommendation Tasks 28013.2.3 Recommendation Techniques 28113.2.3.1 Knowledge-based Techniques 28113.2.3.2 Content-based Techniques 28213.2.3.3 Collaborative Filtering Techniques 28213.2.4 Final Remarks 28913.3 Social Network Analysis 29113.3.1 Representing Social Networks 29113.3.2 Basic Properties of Nodes 29413.3.2.1 Degree 29413.3.2.2 Distance 29413.3.2.3 Closeness 29513.3.2.4 Betweenness 29613.3.2.5 Clustering Coefficient 29713.3.3 Basic and Structural Properties of Networks 29713.3.3.1 Diameter 29713.3.3.2 Centralization 29713.3.3.3 Cliques 29913.3.3.4 Clustering Coefficient 29913.3.3.5 Modularity 29913.3.4 Trends and Final Remarks 29913.4 Exercises 300Apendix A: Comprehensive Description of the CRISP-DM Methodology 303References 311Index 315