Foundations of Average-Cost Nonhomogeneous Controlled Markov Chains

Professor Xi-Ren Cao gained his B.S. in 1967 from the University of Sciences and Technology of China, and his M.S. and Ph.D. from Harvard University, in 1982 and 1984, respectively. He has worked in numerous industrial, teaching, and research positions since then, and is now a Professor Emeritus, The Hong Kong University of Science and Technology. He has acted as an Industry Consultant, was Editor-in-Chief of Discrete Event Dynamic Systems: Theory and Applications for 9 years, and is a Fellow of IFAC and the IEEE. He has published 125 peer-reviewed journal papers, 12 invited book chapters, and four books.

“The book presents a complete and interesting analysis for the optimization of TNHMCs with long-run average performance.” (Raúl Montes-de-Oca, Mathematical Reviews, March, 2022)

• Chapter 1. Introduction.- Chapter 2. Confluencity and State Classification.- Chapter 3. Optimization of Average Rewards and Bias: Single Class.- Chapter 4. Optimization of Average Rewards: Multi-Chains.- Chapter 5. The Nth-Bias and Blackwell Optimality.