JunQi Zhang, PhD, is a Full Professor with Tongji University in Shanghai. He has published 10+ papers in IEEE Transactions and 30+ papers in conferences. His current research interests include learning automata, swarm intelligence, swarm robots, multi-agent systems, reinforcement learning, and big data. MengChu Zhou, PhD, is a Distinguished Professor at New Jersey Institute of Technology. He has over 1100 publications including 14 books, 750+ journal papers (600+ in IEEE transactions), 31 patents, and 32 book-chapters. He is Fellow of IEEE, IFAC, AAAS, CAA and NAI.

• About the Authors ixPreface xiAcknowledgments xiiiA Guide to Reading this Book xvOrganization of the Book xvii1 Introduction 11.1 Ranking and Selection in Noisy Optimization 21.2 Learning Automata and Ordinal Optimization 51.3 Exercises 72 Learning Automata 92.1 Environment and Automaton 92.1.1 Environment 92.1.2 Automaton 102.1.3 Deterministic and Stochastic Automata 112.1.4 Measured Norms 152.2 Fixed Structure Learning Automata 162.2.1 Tsetlin Learning Automaton 162.2.2 Krinsky Learning Automaton 182.2.3 Krylov Learning Automaton 192.2.4 IJA Learning Automaton 202.3 Variable Structure Learning Automata 212.3.1 Estimator-Free Learning Automaton 222.3.2 Deterministic Estimator Learning Automaton 242.3.3 Stochastic Estimator Learning Automaton 262.4 Summary 272.5 Exercises 283 Fast Learning Automata 313.1 Last-position Elimination-based Learning Automata 313.1.1 Background and Motivation 323.1.2 Principles and Algorithm Design 353.1.3 Difference Analysis 373.1.4 Simulation Studies 403.1.5 Summary 453.2 Fast Discretized Pursuit Learning Automata 463.2.1 Background and Motivation 463.2.2 Algorithm Design of Fast Discretized Pursuit LAs 483.2.3 Optimality Analysis 543.2.4 Simulation Studies 593.2.5 Summary 633.3 Exercises 634 Application-Oriented Learning Automata 674.1 Discovering and Tracking Spatiotemporal Event Patterns 674.1.1 Background and Motivation 694.1.2 Spatiotemporal Pattern Learning Automata 704.1.3 Adaptive Tunable Spatiotemporal Pattern Learning Automata 734.1.4 Optimality Analysis 764.1.5 Simulation Studies 834.1.6 Summary 894.2 Stochastic Searching on the Line 894.2.1 Background and Motivation 894.2.2 Symmetrical Hierarchical Stochastic Searching on the Line 954.2.3 Simulation Studies 994.2.4 Summary 1044.3 Fast Adaptive Search on the Line in Dual Environments 1044.3.1 Background and Motivation 1094.3.2 Symmetrized ASS with Buffer 1114.3.3 Simulation Studies 1144.3.4 Summary 1184.4 Exercises 1185 Ordinal Optimization 1235.1 Optimal Computing-Budget Allocation 1235.2 Optimal Computing-Budget Allocation for Selection of Best and Worst Designs 1255.2.1 Background and Motivation 1255.2.2 Approximate Optimal Simulation Budget Allocation 1265.2.3 Simulation Studies 1385.2.4 Summary 1505.3 Optimal Computing-Budget Allocation for Subset Ranking 1515.3.1 Background and Motivation 1515.3.2 Approximate Optimal Simulation Budget Allocation 1535.3.3 Simulation Studies 1595.3.4 Summary 1675.4 Exercises 1676 Incorporation of Ordinal Optimization into Learning Automata 1756.1 Background and Motivation 1756.2 Learning Automata with Optimal Computing Budget Allocation 1786.3 Proof of Optimality 1826.4 Simulation Studies 1876.5 Summary 1936.6 Exercises 1937 Noisy Optimization Applications 1997.1 Background and Motivation 2007.2 Particle Swarm Optimization 2027.2.1 Parameters Configurations 2037.2.2 Topology Structures 2037.2.3 Hybrid PSO 2037.2.4 Multiswarm Techniques 2047.3 Resampling for Noisy Optimization Problems 2047.4 PSO-Based LA and OCBA 2057.5 Simulations Studies 2097.6 Summary 2237.7 Exercises 2248 Applications and Future Research Directions of Learning Automata 2318.1 Summary of Existing Applications 2318.1.1 Classification 2318.1.2 Clustering 2338.1.3 Games 2338.1.4 Knapsack Problems 2348.1.5 Decision Problems in Networks 2358.1.6 Optimization 2368.1.7 LA Parallelization and Design Ranking 2388.1.8 Scheduling 2408.2 Future Research Directions 2418.3 Exercises 243References 243Index 249

JunQi Zhang, PhD, is a Full Professor with Tongji University in Shanghai. He has published 10+ papers in IEEE Transactions and 30+ papers in conferences. His current research interests include learning automata, swarm intelligence, swarm robots, multi-agent systems, reinforcement learning, and big data. MengChu Zhou, PhD, is a Distinguished Professor at New Jersey Institute of Technology. He has over 1100 publications including 14 books, 750+ journal papers (600+ in IEEE transactions), 31 patents, and 32 book-chapters. He is Fellow of IEEE, IFAC, AAAS, CAA and NAI.

• About the Authors ixPreface xiAcknowledgments xiiiA Guide to Reading this Book xvOrganization of the Book xvii1 Introduction 11.1 Ranking and Selection in Noisy Optimization 21.2 Learning Automata and Ordinal Optimization 51.3 Exercises 72 Learning Automata 92.1 Environment and Automaton 92.1.1 Environment 92.1.2 Automaton 102.1.3 Deterministic and Stochastic Automata 112.1.4 Measured Norms 152.2 Fixed Structure Learning Automata 162.2.1 Tsetlin Learning Automaton 162.2.2 Krinsky Learning Automaton 182.2.3 Krylov Learning Automaton 192.2.4 IJA Learning Automaton 202.3 Variable Structure Learning Automata 212.3.1 Estimator-Free Learning Automaton 222.3.2 Deterministic Estimator Learning Automaton 242.3.3 Stochastic Estimator Learning Automaton 262.4 Summary 272.5 Exercises 283 Fast Learning Automata 313.1 Last-position Elimination-based Learning Automata 313.1.1 Background and Motivation 323.1.2 Principles and Algorithm Design 353.1.3 Difference Analysis 373.1.4 Simulation Studies 403.1.5 Summary 453.2 Fast Discretized Pursuit Learning Automata 463.2.1 Background and Motivation 463.2.2 Algorithm Design of Fast Discretized Pursuit LAs 483.2.3 Optimality Analysis 543.2.4 Simulation Studies 593.2.5 Summary 633.3 Exercises 634 Application-Oriented Learning Automata 674.1 Discovering and Tracking Spatiotemporal Event Patterns 674.1.1 Background and Motivation 694.1.2 Spatiotemporal Pattern Learning Automata 704.1.3 Adaptive Tunable Spatiotemporal Pattern Learning Automata 734.1.4 Optimality Analysis 764.1.5 Simulation Studies 834.1.6 Summary 894.2 Stochastic Searching on the Line 894.2.1 Background and Motivation 894.2.2 Symmetrical Hierarchical Stochastic Searching on the Line 954.2.3 Simulation Studies 994.2.4 Summary 1044.3 Fast Adaptive Search on the Line in Dual Environments 1044.3.1 Background and Motivation 1094.3.2 Symmetrized ASS with Buffer 1114.3.3 Simulation Studies 1144.3.4 Summary 1184.4 Exercises 1185 Ordinal Optimization 1235.1 Optimal Computing-Budget Allocation 1235.2 Optimal Computing-Budget Allocation for Selection of Best and Worst Designs 1255.2.1 Background and Motivation 1255.2.2 Approximate Optimal Simulation Budget Allocation 1265.2.3 Simulation Studies 1385.2.4 Summary 1505.3 Optimal Computing-Budget Allocation for Subset Ranking 1515.3.1 Background and Motivation 1515.3.2 Approximate Optimal Simulation Budget Allocation 1535.3.3 Simulation Studies 1595.3.4 Summary 1675.4 Exercises 1676 Incorporation of Ordinal Optimization into Learning Automata 1756.1 Background and Motivation 1756.2 Learning Automata with Optimal Computing Budget Allocation 1786.3 Proof of Optimality 1826.4 Simulation Studies 1876.5 Summary 1936.6 Exercises 1937 Noisy Optimization Applications 1997.1 Background and Motivation 2007.2 Particle Swarm Optimization 2027.2.1 Parameters Configurations 2037.2.2 Topology Structures 2037.2.3 Hybrid PSO 2037.2.4 Multiswarm Techniques 2047.3 Resampling for Noisy Optimization Problems 2047.4 PSO-Based LA and OCBA 2057.5 Simulations Studies 2097.6 Summary 2237.7 Exercises 2248 Applications and Future Research Directions of Learning Automata 2318.1 Summary of Existing Applications 2318.1.1 Classification 2318.1.2 Clustering 2338.1.3 Games 2338.1.4 Knapsack Problems 2348.1.5 Decision Problems in Networks 2358.1.6 Optimization 2368.1.7 LA Parallelization and Design Ranking 2388.1.8 Scheduling 2408.2 Future Research Directions 2418.3 Exercises 243References 243Index 249

JunQi Zhang, PhD, is a Full Professor with Tongji University in Shanghai. He has published 10+ papers in IEEE Transactions and 30+ papers in conferences. His current research interests include learning automata, swarm intelligence, swarm robots, multi-agent systems, reinforcement learning, and big data. MengChu Zhou, PhD, is a Distinguished Professor at New Jersey Institute of Technology. He has over 1100 publications including 14 books, 750+ journal papers (600+ in IEEE transactions), 31 patents, and 32 book-chapters. He is Fellow of IEEE, IFAC, AAAS, CAA and NAI.

• About the Authors ixPreface xiAcknowledgments xiiiA Guide to Reading this Book xvOrganization of the Book xvii1 Introduction 11.1 Ranking and Selection in Noisy Optimization 21.2 Learning Automata and Ordinal Optimization 51.3 Exercises 72 Learning Automata 92.1 Environment and Automaton 92.1.1 Environment 92.1.2 Automaton 102.1.3 Deterministic and Stochastic Automata 112.1.4 Measured Norms 152.2 Fixed Structure Learning Automata 162.2.1 Tsetlin Learning Automaton 162.2.2 Krinsky Learning Automaton 182.2.3 Krylov Learning Automaton 192.2.4 IJA Learning Automaton 202.3 Variable Structure Learning Automata 212.3.1 Estimator-Free Learning Automaton 222.3.2 Deterministic Estimator Learning Automaton 242.3.3 Stochastic Estimator Learning Automaton 262.4 Summary 272.5 Exercises 283 Fast Learning Automata 313.1 Last-position Elimination-based Learning Automata 313.1.1 Background and Motivation 323.1.2 Principles and Algorithm Design 353.1.3 Difference Analysis 373.1.4 Simulation Studies 403.1.5 Summary 453.2 Fast Discretized Pursuit Learning Automata 463.2.1 Background and Motivation 463.2.2 Algorithm Design of Fast Discretized Pursuit LAs 483.2.3 Optimality Analysis 543.2.4 Simulation Studies 593.2.5 Summary 633.3 Exercises 634 Application-Oriented Learning Automata 674.1 Discovering and Tracking Spatiotemporal Event Patterns 674.1.1 Background and Motivation 694.1.2 Spatiotemporal Pattern Learning Automata 704.1.3 Adaptive Tunable Spatiotemporal Pattern Learning Automata 734.1.4 Optimality Analysis 764.1.5 Simulation Studies 834.1.6 Summary 894.2 Stochastic Searching on the Line 894.2.1 Background and Motivation 894.2.2 Symmetrical Hierarchical Stochastic Searching on the Line 954.2.3 Simulation Studies 994.2.4 Summary 1044.3 Fast Adaptive Search on the Line in Dual Environments 1044.3.1 Background and Motivation 1094.3.2 Symmetrized ASS with Buffer 1114.3.3 Simulation Studies 1144.3.4 Summary 1184.4 Exercises 1185 Ordinal Optimization 1235.1 Optimal Computing-Budget Allocation 1235.2 Optimal Computing-Budget Allocation for Selection of Best and Worst Designs 1255.2.1 Background and Motivation 1255.2.2 Approximate Optimal Simulation Budget Allocation 1265.2.3 Simulation Studies 1385.2.4 Summary 1505.3 Optimal Computing-Budget Allocation for Subset Ranking 1515.3.1 Background and Motivation 1515.3.2 Approximate Optimal Simulation Budget Allocation 1535.3.3 Simulation Studies 1595.3.4 Summary 1675.4 Exercises 1676 Incorporation of Ordinal Optimization into Learning Automata 1756.1 Background and Motivation 1756.2 Learning Automata with Optimal Computing Budget Allocation 1786.3 Proof of Optimality 1826.4 Simulation Studies 1876.5 Summary 1936.6 Exercises 1937 Noisy Optimization Applications 1997.1 Background and Motivation 2007.2 Particle Swarm Optimization 2027.2.1 Parameters Configurations 2037.2.2 Topology Structures 2037.2.3 Hybrid PSO 2037.2.4 Multiswarm Techniques 2047.3 Resampling for Noisy Optimization Problems 2047.4 PSO-Based LA and OCBA 2057.5 Simulations Studies 2097.6 Summary 2237.7 Exercises 2248 Applications and Future Research Directions of Learning Automata 2318.1 Summary of Existing Applications 2318.1.1 Classification 2318.1.2 Clustering 2338.1.3 Games 2338.1.4 Knapsack Problems 2348.1.5 Decision Problems in Networks 2358.1.6 Optimization 2368.1.7 LA Parallelization and Design Ranking 2388.1.8 Scheduling 2408.2 Future Research Directions 2418.3 Exercises 243References 243Index 249