Data Analytics for IT Networks

John Garrett is CCIE Emeritus (6204) and Splunk Certified. He earned an M.S. in predictive analytics from Northwestern University, and has a patent pending related to analysis of network devices with data science techniques. John has architected, designed, and implemented LAN, WAN, wireless, and data center solutions for some of the largest Cisco customers. As a secondary role, John has worked with teams in the Cisco Services organization to innovate on some of the most widely used tools and methodologies at Customer Experience over the past 12 years.For the past 7 years, John’s journey has moved through server virtualization, network virtualization, OpenStack and cloud, network functions virtualization (NFV), service assurance, and data science. The realization that analytics and data science play roles in all these brought John full circle back to developing innovative tools and techniques for Cisco Services. John’s most recent role is as an Analytics Technical Lead, developing use cases to benefit Cisco Services customers as part of Business Critical Services for Cisco. John lives with his wife and children in Raleigh, North Carolina.

• Foreword xviiIntroduction: Your future is in your hands! xviiiChapter 1 Getting Started with Analytics 1What This Chapter Covers 1Data: You as the SME 2Use-Case Development with Bias and Mental Models 2Data Science: Algorithms and Their Purposes 3What This Book Does Not Cover 4Building a Big Data Architecture 4Microservices Architectures and Open Source Software 5R Versus Python Versus SAS Versus Stata 6Databases and Data Storage 6Cisco Products in Detail 6Analytics and Literary Perspectives 7Analytics Maturity 7Knowledge Management 8Gartner Analytics 8Strategic Thinking 9Striving for “Up and to the Right” 9Moving Your Perspective 10Hot Topics in the Literature 11Summary 12Chapter 2 Approaches for Analytics and Data Science 13Model Building and Model Deployment 14Analytics Methodology and Approach 15Common Approach Walkthrough 16Distinction Between the Use Case and the Solution 18Logical Models for Data Science and Data 19Analytics as an Overlay 20Analytics Infrastructure Model 22Summary 33Chapter 3 Understanding Networking Data Sources 35Planes of Operation on IT Networks 36Review of the Planes 40Data and the Planes of Operation 42Planes Data Examples 44A Wider Rabbit Hole 49A Deeper Rabbit Hole 51Summary 53Chapter 4 Accessing Data from Network Components 55Methods of Networking Data Access 55Pull Data Availability 57Push Data Availability 61Control Plane Data 67Data Plane Traffic Capture 68Packet Data 70Other Data Access Methods 74Data Types and Measurement Considerations 76Numbers and Text 77Data Structure 82Data Manipulation 84Other Data Considerations 87External Data for Context 89Data Transport Methods 89Transport Considerations for Network Data Sources 90Summary 96Chapter 5 Mental Models and Cognitive Bias 97Changing How You Think 98Domain Expertise, Mental Models, and Intuition 99Mental Models 99Daniel Kahneman’s System 1 and System 2 102Intuition 103Opening Your Mind to Cognitive Bias 104Changing Perspective, Using Bias for Good 105Your Bias and Your Solutions 106How You Think: Anchoring, Focalism, Narrative Fallacy, Framing, and Priming 107How Others Think: Mirroring 110What Just Happened? Availability, Recency, Correlation, Clustering, and Illusion of Truth 111Enter the Boss: HIPPO and Authority Bias 113What You Know: Confirmation, Expectation, Ambiguity, Context, and Frequency Illusion 114What You Don’t Know: Base Rates, Small Numbers, Group Attribution, and Survivorship 117Your Skills and Expertise: Curse of Knowledge, Group Bias, and Dunning-Kruger 119We Don’t Need a New System: IKEA, Not Invented Here, Pro-Innovation, Endowment, Status Quo, Sunk Cost, Zero Price, and Empathy 121I Knew It Would Happen: Hindsight, Halo Effect, and Outcome Bias 123Summary 124Chapter 6 Innovative Thinking Techniques 127Acting Like an Innovator and Mindfulness 128Innovation Tips and Techniques 129Developing Analytics for Your Company 140Defocusing, Breaking Anchors, and Unpriming 140Lean Thinking 142Cognitive Trickery 143Quick Innovation Wins 143Summary 144Chapter 7 Analytics Use Cases and the Intuition Behind Them 147Analytics Definitions 150How to Use the Information from This Chapter 151Priming and Framing Effects 151Analytics Rube Goldberg Machines 151Popular Analytics Use Cases 152Machine Learning and Statistics Use Cases 153Common IT Analytics Use Cases 170Broadly Applicable Use Cases 199Some Final Notes on Use Cases 214Summary 214Chapter 8 Analytics Algorithms and the Intuition Behind Them 217About the Algorithms 217Algorithms and Assumptions 218Additional Background 219Data and Statistics 221Statistics 221Correlation 224Longitudinal Data 225ANOVA 227Probability 228Bayes’ Theorem 228Feature Selection 230Data-Encoding Methods 232Dimensionality Reduction 233Unsupervised Learning 234Clustering 234Association Rules 240Sequential Pattern Mining 243Collaborative Filtering 244Supervised Learning 246Regression Analysis 246Classification Algorithms 248Decision Trees 249Random Forest 250Gradient Boosting Methods 251Neural Networks 252Support Vector Machines 258Time Series Analysis 259Text and Document Analysis 262Natural Language Processing (NLP) 262Information Retrieval 263Topic Modeling 265Sentiment Analysis 266Other Analytics Concepts 267Artificial Intelligence 267Confusion Matrix and Contingency Tables 267Cumulative Gains and Lift 269Simulation 271Summary 271Chapter 9 Building Analytics Use Cases 273Designing Your Analytics Solutions 274Using the Analytics Infrastructure Model 275About the Upcoming Use Cases 276The Data 276The Data Science 278The Code 280Operationalizing Solutions as Use Cases 281Understanding and Designing Workflows 282Tips for Setting Up an Environment to Do Your Own Analysis 282Summary 284Chapter 10 Developing Real Use Cases: The Power of Statistics 285Loading and Exploring Data 286Base Rate Statistics for Platform Crashes 288Base Rate Statistics for Software Crashes 299ANOVA 305Data Transformation 310Tests for Normality 311Examining Variance 313Statistical Anomaly Detection 318Summary 321Chapter 11 Developing Real Use Cases: Network Infrastructure Analytics 323Human DNA and Fingerprinting 324Building Search Capability 325Loading Data and Setting Up the Environment 325Encoding Data for Algorithmic Use 328Search Challenges and Solutions 331Other Uses of Encoded Data 336Dimensionality Reduction 337Data Visualization 340K-Means Clustering 344Machine Learning Guided Troubleshooting 350Summary 353Chapter 12 Developing Real Use Cases: Control Plane Analytics Using Syslog Telemetry 355Data for This Chapter 356OSPF Routing Protocols 357Non-Machine Learning Log Analysis Using pandas 357Noise Reduction 360Finding the Hotspots 362Machine Learning—Based Log Evaluation 366Data Visualization 367Cleaning and Encoding Data 369Clustering 373More Data Visualization 375Transaction Analysis 379Task List 386Summary 387Chapter 13 Developing Real Use Cases: Data Plane Analytics 389The Data 390SME Analysis 394SME Port Clustering 407Machine Learning: Creating Full Port Profiles 413Machine Learning: Creating Source Port Profiles 419Asset Discovery 422Investigation Task List 423Summary 424Chapter 14 Cisco Analytics 425Architecture and Advisory Services for Analytics 426Stealthwatch 427Digital Network Architecture (DNA) 428AppDynamics 428Tetration 430Crosswork Automation 431IoT Analytics 432Analytics Platforms and Partnerships 433Cisco Open Source Platform 433Summary 434Chapter 15 Book Summary 435Analytics Introduction and Methodology 436All About Networking Data 438Using Bias and Innovation to Discover Solutions 439Analytics Use Cases and Algorithms 439Building Real Analytics Use Cases 440Cisco Services and Solutions 442In Closing 442Appendix A Function for Parsing Packets from pcap Files 4439781587145131, TOC, 9/19/18