James Keller holds the University of Missouri Curators' Professorship in the Electrical and Computer Engineering and Computer Science Departments on the Columbia Campus, and is the R.L. Tatum Professor in the College of Engineering. Dr. Keller is a Life Fellow of the IEEE, a Fellow of the International Fuzzy Systems Association, and a former president of the North American Fuzzy Information Processing Society.Derong Liu is a Professor of Electrical and Computer Engineering at the University of Illinois at Chicago, USA, and a Professor of Automation and Electrical Engineering at the University of Science and Technology Beijing, China. Dr. Liu is a Fellow of the IEEE and a Fellow of the International Neural Network Society. He has published 17 books, including Reinforcement Learning and Approximate Dynamic Programming for Feedback Control (2012, Wiley-IEEE Press). He is the Editor-in-Chief of Artificial Intelligence Review, and he served as the Editor-in-Chief of the IEEE Transactions on Neural Networks and Learning Systems (2010-2015).David Fogel is the President of Natural Selection, Inc., CEO of Natural Selection Financial, Inc., a Fellow of the IEEE, and the series editor for the Wiley-IEEE Press Series on Computational Intelligence. Dr. Fogel has 30 years of experience pioneering contributions in the field of computational intelligence, and is co-inventor of the EffectCheck® sentiment analysis system. He has written several books including Evolutionary Computation: The Fossil Record (1998) and Evolutionary Computation Toward a New Philosophy of Machine Intelligence, 3rd Edition (2005), both published by the Wiley-IEEE Press.

• Acknowledgments xi1. Introduction to Computational Intelligence 11.1 Welcome to Computational Intelligence 11.2 What Makes This Book Special 11.3 What This Book Covers 21.4 How to Use This Book 21.5 Final Thoughts Before You Get Started 3PART I NEURAL NETWORKS 52. Introduction and Single-Layer Neural Networks 72.1 Short History of Neural Networks 92.2 Rosenblatt’s Neuron 102.3 Perceptron Training Algorithm 132.4 The Perceptron Convergence Theorem 232.5 Computer Experiment Using Perceptrons 252.6 Activation Functions 28Exercises 303. Multilayer Neural Networks and Backpropagation 353.1 Universal Approximation Theory 353.2 The Backpropagation Training Algorithm 373.3 Batch Learning and Online Learning 453.4 Cross-Validation and Generalization 473.5 Computer Experiment Using Backpropagation 53Exercises 564. Radial-Basis Function Networks 614.1 Radial-Basis Functions 614.2 The Interpolation Problem 624.3 Training Algorithms For Radial-Basis Function Networks 644.4 Universal Approximation 694.5 Kernel Regression 70Exercises 755. Recurrent Neural Networks 775.1 The Hopfield Network 775.2 The Grossberg Network 815.3 Cellular Neural Networks 885.4 Neurodynamics and Optimization 915.5 Stability Analysis of Recurrent Neural Networks 93Exercises 99PART II FUZZY SET THEORY AND FUZZY LOGIC 1016. Basic Fuzzy Set Theory 1036.1 Introduction 1036.2 A Brief History 1076.3 Fuzzy Membership Functions and Operators 1086.4 Alpha-Cuts, The Decomposition Theorem, and The Extension Principle 1176.5 Compensatory Operators 1206.6 Conclusions 124Exercises 1247. Fuzzy Relations and Fuzzy Logic Inference 1277.1 Introduction 1277.2 Fuzzy Relations and Propositions 1287.3 Fuzzy Logic Inference 1317.4 Fuzzy Logic For Real-Valued Inputs 1357.5 Where Do The Rules Come From? 1387.6 Chapter Summary 142Exercises 1438. Fuzzy Clustering and Classification 1478.1 Introduction to Fuzzy Clustering 1478.2 Fuzzy c-Means 1558.3 An Extension of The Fuzzy c-Means 1678.4 Possibilistic c-Means 1698.5 Fuzzy Classifiers: Fuzzy k-Nearest Neighbors 1748.6 Chapter Summary 179Exercises 1809. Fuzzy Measures and Fuzzy Integrals 1839.1 Fuzzy Measures 1839.2 Fuzzy Integrals 1889.3 Training The Fuzzy Integrals 1919.4 Summary and Final Thoughts 203Exercises 203PART III EVOLUTIONARY COMPUTATION 20710. Evolutionary Computation 20910.1 Basic Ideas and Fundamentals 20910.2 Evolutionary Algorithms: Generate and Test 21610.3 Representation, Search, and Selection Operators 22110.4 Major Research and Application Areas 22310.5 Summary 225Exercises 22511. Evolutionary Optimization 22711.1 Global Numerical Optimization 22911.2 Combinatorial Optimization 23311.3 Some Mathematical Considerations 23811.4 Constraint Handling 25511.5 Self-Adaptation 25811.6 Summary 264Exercises 26512. Evolutionary Learning and Problem Solving 26912.1 Evolving Parameters of A Regression Equation 27012.2 Evolving The Structure and Parameters of Input–Output Systems 27412.3 Evolving Clusters 29212.4 Evolutionary Classification Models 29812.5 Evolutionary Control Systems 30712.6 Evolutionary Games 31412.7 Summary 320Exercises 32113. Collective Intelligence and Other Extensions of Evolutionary Computation 32313.1 Particle Swarm Optimization 32313.2 Differential Evolution 32613.3 Ant Colony Optimization 32913.4 Evolvable Hardware 33113.5 Interactive Evolutionary Computation 33313.6 Multicriteria Evolutionary Optimization 33513.7 Summary 340Exercises 340References 343Index 361

James Keller holds the University of Missouri Curators' Professorship in the Electrical and Computer Engineering and Computer Science Departments on the Columbia Campus, and is the R.L. Tatum Professor in the College of Engineering. Dr. Keller is a Life Fellow of the IEEE, a Fellow of the International Fuzzy Systems Association, and a former president of the North American Fuzzy Information Processing Society.Derong Liu is a Professor of Electrical and Computer Engineering at the University of Illinois at Chicago, USA, and a Professor of Automation and Electrical Engineering at the University of Science and Technology Beijing, China. Dr. Liu is a Fellow of the IEEE and a Fellow of the International Neural Network Society. He has published 17 books, including Reinforcement Learning and Approximate Dynamic Programming for Feedback Control (2012, Wiley-IEEE Press). He is the Editor-in-Chief of Artificial Intelligence Review, and he served as the Editor-in-Chief of the IEEE Transactions on Neural Networks and Learning Systems (2010-2015).David Fogel is the President of Natural Selection, Inc., CEO of Natural Selection Financial, Inc., a Fellow of the IEEE, and the series editor for the Wiley-IEEE Press Series on Computational Intelligence. Dr. Fogel has 30 years of experience pioneering contributions in the field of computational intelligence, and is co-inventor of the EffectCheck® sentiment analysis system. He has written several books including Evolutionary Computation: The Fossil Record (1998) and Evolutionary Computation Toward a New Philosophy of Machine Intelligence, 3rd Edition (2005), both published by the Wiley-IEEE Press.

• Acknowledgments xi1. Introduction to Computational Intelligence 11.1 Welcome to Computational Intelligence 11.2 What Makes This Book Special 11.3 What This Book Covers 21.4 How to Use This Book 21.5 Final Thoughts Before You Get Started 3PART I NEURAL NETWORKS 52. Introduction and Single-Layer Neural Networks 72.1 Short History of Neural Networks 92.2 Rosenblatt’s Neuron 102.3 Perceptron Training Algorithm 132.4 The Perceptron Convergence Theorem 232.5 Computer Experiment Using Perceptrons 252.6 Activation Functions 28Exercises 303. Multilayer Neural Networks and Backpropagation 353.1 Universal Approximation Theory 353.2 The Backpropagation Training Algorithm 373.3 Batch Learning and Online Learning 453.4 Cross-Validation and Generalization 473.5 Computer Experiment Using Backpropagation 53Exercises 564. Radial-Basis Function Networks 614.1 Radial-Basis Functions 614.2 The Interpolation Problem 624.3 Training Algorithms For Radial-Basis Function Networks 644.4 Universal Approximation 694.5 Kernel Regression 70Exercises 755. Recurrent Neural Networks 775.1 The Hopfield Network 775.2 The Grossberg Network 815.3 Cellular Neural Networks 885.4 Neurodynamics and Optimization 915.5 Stability Analysis of Recurrent Neural Networks 93Exercises 99PART II FUZZY SET THEORY AND FUZZY LOGIC 1016. Basic Fuzzy Set Theory 1036.1 Introduction 1036.2 A Brief History 1076.3 Fuzzy Membership Functions and Operators 1086.4 Alpha-Cuts, The Decomposition Theorem, and The Extension Principle 1176.5 Compensatory Operators 1206.6 Conclusions 124Exercises 1247. Fuzzy Relations and Fuzzy Logic Inference 1277.1 Introduction 1277.2 Fuzzy Relations and Propositions 1287.3 Fuzzy Logic Inference 1317.4 Fuzzy Logic For Real-Valued Inputs 1357.5 Where Do The Rules Come From? 1387.6 Chapter Summary 142Exercises 1438. Fuzzy Clustering and Classification 1478.1 Introduction to Fuzzy Clustering 1478.2 Fuzzy c-Means 1558.3 An Extension of The Fuzzy c-Means 1678.4 Possibilistic c-Means 1698.5 Fuzzy Classifiers: Fuzzy k-Nearest Neighbors 1748.6 Chapter Summary 179Exercises 1809. Fuzzy Measures and Fuzzy Integrals 1839.1 Fuzzy Measures 1839.2 Fuzzy Integrals 1889.3 Training The Fuzzy Integrals 1919.4 Summary and Final Thoughts 203Exercises 203PART III EVOLUTIONARY COMPUTATION 20710. Evolutionary Computation 20910.1 Basic Ideas and Fundamentals 20910.2 Evolutionary Algorithms: Generate and Test 21610.3 Representation, Search, and Selection Operators 22110.4 Major Research and Application Areas 22310.5 Summary 225Exercises 22511. Evolutionary Optimization 22711.1 Global Numerical Optimization 22911.2 Combinatorial Optimization 23311.3 Some Mathematical Considerations 23811.4 Constraint Handling 25511.5 Self-Adaptation 25811.6 Summary 264Exercises 26512. Evolutionary Learning and Problem Solving 26912.1 Evolving Parameters of A Regression Equation 27012.2 Evolving The Structure and Parameters of Input–Output Systems 27412.3 Evolving Clusters 29212.4 Evolutionary Classification Models 29812.5 Evolutionary Control Systems 30712.6 Evolutionary Games 31412.7 Summary 320Exercises 32113. Collective Intelligence and Other Extensions of Evolutionary Computation 32313.1 Particle Swarm Optimization 32313.2 Differential Evolution 32613.3 Ant Colony Optimization 32913.4 Evolvable Hardware 33113.5 Interactive Evolutionary Computation 33313.6 Multicriteria Evolutionary Optimization 33513.7 Summary 340Exercises 340References 343Index 361

James Keller holds the University of Missouri Curators' Professorship in the Electrical and Computer Engineering and Computer Science Departments on the Columbia Campus, and is the R.L. Tatum Professor in the College of Engineering. Dr. Keller is a Life Fellow of the IEEE, a Fellow of the International Fuzzy Systems Association, and a former president of the North American Fuzzy Information Processing Society.Derong Liu is a Professor of Electrical and Computer Engineering at the University of Illinois at Chicago, USA, and a Professor of Automation and Electrical Engineering at the University of Science and Technology Beijing, China. Dr. Liu is a Fellow of the IEEE and a Fellow of the International Neural Network Society. He has published 17 books, including Reinforcement Learning and Approximate Dynamic Programming for Feedback Control (2012, Wiley-IEEE Press). He is the Editor-in-Chief of Artificial Intelligence Review, and he served as the Editor-in-Chief of the IEEE Transactions on Neural Networks and Learning Systems (2010-2015).David Fogel is the President of Natural Selection, Inc., CEO of Natural Selection Financial, Inc., a Fellow of the IEEE, and the series editor for the Wiley-IEEE Press Series on Computational Intelligence. Dr. Fogel has 30 years of experience pioneering contributions in the field of computational intelligence, and is co-inventor of the EffectCheck® sentiment analysis system. He has written several books including Evolutionary Computation: The Fossil Record (1998) and Evolutionary Computation Toward a New Philosophy of Machine Intelligence, 3rd Edition (2005), both published by the Wiley-IEEE Press.

• Acknowledgments xi1. Introduction to Computational Intelligence 11.1 Welcome to Computational Intelligence 11.2 What Makes This Book Special 11.3 What This Book Covers 21.4 How to Use This Book 21.5 Final Thoughts Before You Get Started 3PART I NEURAL NETWORKS 52. Introduction and Single-Layer Neural Networks 72.1 Short History of Neural Networks 92.2 Rosenblatt’s Neuron 102.3 Perceptron Training Algorithm 132.4 The Perceptron Convergence Theorem 232.5 Computer Experiment Using Perceptrons 252.6 Activation Functions 28Exercises 303. Multilayer Neural Networks and Backpropagation 353.1 Universal Approximation Theory 353.2 The Backpropagation Training Algorithm 373.3 Batch Learning and Online Learning 453.4 Cross-Validation and Generalization 473.5 Computer Experiment Using Backpropagation 53Exercises 564. Radial-Basis Function Networks 614.1 Radial-Basis Functions 614.2 The Interpolation Problem 624.3 Training Algorithms For Radial-Basis Function Networks 644.4 Universal Approximation 694.5 Kernel Regression 70Exercises 755. Recurrent Neural Networks 775.1 The Hopfield Network 775.2 The Grossberg Network 815.3 Cellular Neural Networks 885.4 Neurodynamics and Optimization 915.5 Stability Analysis of Recurrent Neural Networks 93Exercises 99PART II FUZZY SET THEORY AND FUZZY LOGIC 1016. Basic Fuzzy Set Theory 1036.1 Introduction 1036.2 A Brief History 1076.3 Fuzzy Membership Functions and Operators 1086.4 Alpha-Cuts, The Decomposition Theorem, and The Extension Principle 1176.5 Compensatory Operators 1206.6 Conclusions 124Exercises 1247. Fuzzy Relations and Fuzzy Logic Inference 1277.1 Introduction 1277.2 Fuzzy Relations and Propositions 1287.3 Fuzzy Logic Inference 1317.4 Fuzzy Logic For Real-Valued Inputs 1357.5 Where Do The Rules Come From? 1387.6 Chapter Summary 142Exercises 1438. Fuzzy Clustering and Classification 1478.1 Introduction to Fuzzy Clustering 1478.2 Fuzzy c-Means 1558.3 An Extension of The Fuzzy c-Means 1678.4 Possibilistic c-Means 1698.5 Fuzzy Classifiers: Fuzzy k-Nearest Neighbors 1748.6 Chapter Summary 179Exercises 1809. Fuzzy Measures and Fuzzy Integrals 1839.1 Fuzzy Measures 1839.2 Fuzzy Integrals 1889.3 Training The Fuzzy Integrals 1919.4 Summary and Final Thoughts 203Exercises 203PART III EVOLUTIONARY COMPUTATION 20710. Evolutionary Computation 20910.1 Basic Ideas and Fundamentals 20910.2 Evolutionary Algorithms: Generate and Test 21610.3 Representation, Search, and Selection Operators 22110.4 Major Research and Application Areas 22310.5 Summary 225Exercises 22511. Evolutionary Optimization 22711.1 Global Numerical Optimization 22911.2 Combinatorial Optimization 23311.3 Some Mathematical Considerations 23811.4 Constraint Handling 25511.5 Self-Adaptation 25811.6 Summary 264Exercises 26512. Evolutionary Learning and Problem Solving 26912.1 Evolving Parameters of A Regression Equation 27012.2 Evolving The Structure and Parameters of Input–Output Systems 27412.3 Evolving Clusters 29212.4 Evolutionary Classification Models 29812.5 Evolutionary Control Systems 30712.6 Evolutionary Games 31412.7 Summary 320Exercises 32113. Collective Intelligence and Other Extensions of Evolutionary Computation 32313.1 Particle Swarm Optimization 32313.2 Differential Evolution 32613.3 Ant Colony Optimization 32913.4 Evolvable Hardware 33113.5 Interactive Evolutionary Computation 33313.6 Multicriteria Evolutionary Optimization 33513.7 Summary 340Exercises 340References 343Index 361