Reinforcement and Systemic Machine Learning for Decision Making

Parag Kulkarni, PhD, DSc, is the founder and Chief Scientist of EKLat Research where he has empowered businesses through machine learning, knowledge management, and systemic management. He has been working within the IT industry for over twenty years. The recipient of several awards, Dr. Kulkarni is a pioneer in the field. His areas of research and product development include M-maps, intelligent systems, text mining, image processing, decision systems, forecasting, IT strategy, artificial intelligence, and machine learning. Dr. Kulkarni has over 100 research publications including several books.

• Preface xv Acknowledgments xixAbout the Author xxi1 Introduction to Reinforcement and Systemic Machine Learning 11.1. Introduction 11.2. Supervised, Unsupervised, and Semisupervised Machine Learning 21.3. Traditional Learning Methods and History of Machine Learning 41.4. What Is Machine Learning? 71.5. Machine-Learning Problem 81.6. Learning Paradigms 91.7. Machine-Learning Techniques and Paradigms 121.8. What Is Reinforcement Learning? 141.9. Reinforcement Function and Environment Function 161.10. Need of Reinforcement Learning 171.11. Reinforcement Learning and Machine Intelligence 171.12. What Is Systemic Learning? 181.13. What Is Systemic Machine Learning? 181.14. Challenges in Systemic Machine Learning 191.15. Reinforcement Machine Learning and Systemic Machine Learning 191.16. Case Study Problem Detection in a Vehicle 201.17. Summary 202 Fundamentals of Whole-System, Systemic, and Multiperspective Machine Learning 232.1. Introduction 232.2. What Is Systemic Machine Learning? 272.3. Generalized Systemic Machine-Learning Framework 302.4. Multiperspective Decision Making and Multiperspective Learning 332.5. Dynamic and Interactive Decision Making 432.6. The Systemic Learning Framework 472.7. System Analysis 522.8. Case Study: Need of Systemic Learning in the Hospitality Industry 542.9. Summary 553 Reinforcement Learning 573.1. Introduction 573.2. Learning Agents 603.3. Returns and Reward Calculations 623.4. Reinforcement Learning and Adaptive Control 633.5. Dynamic Systems 663.6. Reinforcement Learning and Control 683.7. Markov Property and Markov Decision Process 683.8. Value Functions 693.8.1. Action and Value 703.9. Learning an Optimal Policy (Model-Based and Model-Free Methods) 703.10. Dynamic Programming 713.11. Adaptive Dynamic Programming 713.12. Example: Reinforcement Learning for Boxing Trainer 753.13. Summary 754 Systemic Machine Learning and Model 774.1. Introduction 774.2. A Framework for Systemic Learning 784.3. Capturing the Systemic View 864.4. Mathematical Representation of System Interactions 894.5. Impact Function 914.6. Decision-Impact Analysis 914.7. Summary 975 Inference and Information Integration 995.1. Introduction 995.2. Inference Mechanisms and Need 1015.3. Integration of Context and Inference 1075.4. Statistical Inference and Induction 1115.5. Pure Likelihood Approach 1125.6. Bayesian Paradigm and Inference 1135.7. Time-Based Inference 1145.8. Inference to Build a System View 1145.9. Summary 1186 Adaptive Learning 1196.1. Introduction 1196.2. Adaptive Learning and Adaptive Systems 1196.3. What Is Adaptive Machine Learning? 1236.4. Adaptation and Learning Method Selection Based on Scenario 1246.5. Systemic Learning and Adaptive Learning 1276.6. Competitive Learning and Adaptive Learning 1406.7. Examples 1466.8. Summary 1497 Multiperspective and Whole-System Learning 1517.1. Introduction 1517.2. Multiperspective Context Building 1527.3. Multiperspective Decision Making and Multiperspective Learning 1547.4. Whole-System Learning and Multiperspective Approaches 1647.5. Case Study Based on Multiperspective Approach 1677.6. Limitations to a Multiperspective Approach 1747.7. Summary 1748 Incremental Learning and Knowledge Representation 1778.1. Introduction 1778.2. Why Incremental Learning? 1788.3. Learning from What Is Already Learned. . . 1808.4. Supervised Incremental Learning 1918.5. Incremental Unsupervised Learning and Incremental Clustering 1918.6. Semisupervised Incremental Learning 1968.7. Incremental and Systemic Learning 1998.8. Incremental Closeness Value and Learning Method 2008.9. Learning and Decision-Making Model 2058.10. Incremental Classification Techniques 2068.11. Case Study: Incremental Document Classification 2078.12. Summary 2089 Knowledge Augmentation: A Machine Learning Perspective 2099.1. Introduction 2099.2. Brief History and Related Work 2119.3. Knowledge Augmentation and Knowledge Elicitation 2159.4. Life Cycle of Knowledge 2179.5. Incremental Knowledge Representation 2229.6. Case-Based Learning and Learning with Reference to Knowledge Loss 2249.7. Knowledge Augmentation: Techniques and Methods 2249.8. Heuristic Learning 2289.9. Systemic Machine Learning and Knowledge Augmentation 2299.10. Knowledge Augmentation in Complex Learning Scenarios 2329.11. Case Studies 2329.12. Summary 23510 Building a Learning System 23710.1. Introduction 23710.2. Systemic Learning System 23710.3. Algorithm Selection 24210.4. Knowledge Representation 24410.5. Designing a Learning System 24510.6. Making System to Behave Intelligently 24610.7. Example-Based Learning 24610.8. Holistic Knowledge Framework and Use of Reinforcement Learning 24610.9. Intelligent Agents—Deployment and Knowledge Acquisition and Reuse 25010.10. Case-Based Learning: Human Emotion-Detection System 25110.11. Holistic View in Complex Decision Problem 25310.12. Knowledge Representation and Data Discovery 25510.13. Components 25810.14. Future of Learning Systems and Intelligent Systems 25910.15. Summary 259Appendix A: Statistical Learning Methods 261Appendix B: Markov Processes 271Index 281