Autonomous Learning Systems

Plamen Parvanov Angelov, Lancaster University, UKPlamen Parvanov is a senior lecturer in the School of Computing and Communications at Lancaster University. He is an Associate Editor of three international journals and the founding co-Editor-in-Chief of the Springer journal Evolving Systems. He is also the Vice Chair of the Technical Committee on Standards, Computational Intelligence Society, IEEE and co-Chair of several IEEE conferences. His research in UAV/UAS is often publicised in external publications, e.g. the prestigious Computational Intelligence Magazine; Aviation Week, Flight Global, Airframer, Flight International, etc. His research focuses on computational intelligence and evolving systems, and his research in to autonomous systems has received worldwide recognition. As the Principle Investigator at Lancaster University for a team working on UAV Sense and Avoid fortwo projects of ASTRAEA his work was recognised by 'The Engineer Innovation and Technology 2008 Award in two categories: i) Aerospace and Defence and ii) The Special Award which is an outstanding achievement.

“Overall, this book presents a valuable framework for further investigation and development for researchers and software developers. Summing Up: Recommended. Graduate students and above.”  (Choice, 1 October 2013)

• Forewords xiPreface xixAbout the Author xxiii1 Introduction 11.1 Autonomous Systems 31.2 The Role of Machine Learning in Autonomous Systems 41.3 System Identification – an Abstract Model of the Real World 61.4 Online versus Offline Identification 91.5 Adaptive and Evolving Systems 101.6 Evolving or Evolutionary Systems 111.7 Supervised versus Unsupervised Learning 131.8 Structure of the Book 14PART I FUNDAMENTALS2 Fundamentals of Probability Theory 192.1 Randomness and Determinism 202.2 Frequentistic versus Belief-Based Approach 222.3 Probability Densities and Moments 232.4 Density Estimation – Kernel-Based Approach 262.5 Recursive Density Estimation (RDE) 282.6 Detecting Novelties/Anomalies/Outliers using RDE 322.7 Conclusions 363 Fundamentals of Machine Learning and Pattern Recognition 373.1 Preprocessing 373.2 Clustering 423.3 Classification 563.4 Conclusions 584 Fundamentals of Fuzzy Systems Theory 614.1 Fuzzy Sets 614.2 Fuzzy Systems, Fuzzy Rules 644.3 Fuzzy Systems with Nonparametric Antecedents (AnYa) 694.4 FRB (Offline) Classifiers 734.5 Neurofuzzy Systems 754.6 State Space Perspective 794.7 Conclusions 81PART II METHODOLOGY OF AUTONOMOUS LEARNING SYSTEMS5 Evolving System Structure from Streaming Data 855.1 Defining System Structure Based on Prior Knowledge 855.2 Data Space Partitioning 865.3 Normalisation and Standardisation of Streaming Data in an Evolving Environment 965.4 Autonomous Monitoring of the Structure Quality 985.5 Short- and Long-Term Focal Points and Submodels 1045.6 Simplification and Interpretability Issues 1055.7 Conclusions 1076 Autonomous Learning Parameters of the Local Submodels 1096.1 Learning Parameters of Local Submodels 1106.2 Global versus Local Learning 1116.3 Evolving Systems Structure Recursively 1136.4 Learning Modes 1166.5 Robustness to Outliers in Autonomous Learning 1186.6 Conclusions 1187 Autonomous Predictors, Estimators, Filters, Inferential Sensors 1217.1 Predictors, Estimators, Filters – Problem Formulation 1217.2 Nonlinear Regression 1237.3 Time Series 1247.4 Autonomous Learning Sensors 1257.5 Conclusions 1318 Autonomous Learning Classifiers 1338.1 Classifying Data Streams 1338.2 Why Adapt the Classifier Structure? 1348.3 Architecture of Autonomous Classifiers of the Family AutoClassify 1358.4 Learning AutoClassify from Streaming Data 1398.5 Analysis of AutoClassify 1408.6 Conclusions 1409 Autonomous Learning Controllers 1439.1 Indirect Adaptive Control Scheme 1449.2 Evolving Inverse Plant Model from Online Streaming Data 1459.3 Evolving Fuzzy Controller Structure from Online Streaming Data 1479.4 Examples of Using AutoControl 1489.5 Conclusions 15310 Collaborative Autonomous Learning Systems 15510.1 Distributed Intelligence Scenarios 15510.2 Autonomous Collaborative Learning 15710.3 Collaborative Autonomous Clustering, AutoCluster by a Team of ALSs 15810.4 Collaborative Autonomous Predictors, Estimators, Filters and AutoSense by a Team of ALSs 15910.5 Collaborative Autonomous Classifiers AutoClassify by a Team of ALSs 16010.6 Superposition of Local Submodels 16110.7 Conclusions 161PART III APPLICATIONS OF ALS11 Autonomous Learning Sensors for Chemical and Petrochemical Industries 16511.1 Case Study 1: Quality of the Products in an Oil Refinery 16511.2 Case Study 2: Polypropylene Manufacturing 17211.3 Conclusions 17812 Autonomous Learning Systems in Mobile Robotics 17912.1 The Mobile Robot Pioneer 3DX 17912.2 Autonomous Classifier for Landmark Recognition 18012.3 Autonomous Leader Follower 19312.4 Results Analysis 19613 Autonomous Novelty Detection and Object Tracking in Video Streams 19713.1 Problem Definition 19713.2 Background Subtraction and KDE for Detecting Visual Novelties 19813.3 Detecting Visual Novelties with the RDE Method 20313.4 Object Identification in Image Frames Using RDE 20413.5 Real-time Tracking in Video Streams Using ALS 20613.6 Conclusions 20914 Modelling Evolving User Behaviour with ALS 21114.1 User Behaviour as an Evolving Phenomenon 21114.2 Designing the User Behaviour Profile 21214.3 Applying AutoClassify0 for Modelling Evolving User Behaviour 21514.4 Case Studies 21614.5 Conclusions 22115 Epilogue 22315.1 Conclusions 22315.2 Open Problems 22715.3 Future Directions 227APPENDICESAppendix A Mathematical Foundations 231Appendix B Pseudocode of the Basic Algorithms 235References 245Glossary 259Index 263