Xi Ren Cao - Böcker

Performance optimization is vital in the design and operation of modern engineering systems, including communications, manufacturing, robotics, and logistics. Most engineering systems are too complicated to model, or the system parameters cannot be easily identified, so learning techniques have to be applied. This is a multi-disciplinary area which has been attracting wide attention across many disciplines. Areas such as perturbation analysis (PA) in discrete event dynamic systems (DEDSs), Markov decision processes (MDPs) in operations research, reinforcement learning (RL) or neuro-dynamic programming (NDP) in computer science, identification and adaptive control (I&AC) in control systems, share the common goal: to make the "best decision" to optimize system performance.This book provides a unified framework based on a sensitivity point of view. It also introduces new approaches and proposes new research topics within this sensitivity-based framework.

Dynamic Systems (DEDS) are almost endless: military C31 Ilogistic systems, the emergency ward of a metropolitan hospital, back offices of large insurance and brokerage fums, service and spare part operations of multinational fums . . . . the point is the pervasive nature of such systems in the daily life of human beings. Yet DEDS is a relatively new phenomenon in dynamic systems studies. From the days of Galileo to Newton to quantum mechanics and cosmology of the present, dynamic systems in nature are primarily differential equations based and time driven. A large literature and endless success stories have been built up on such Continuous Variable Dynamic Systems (CVDS). It is, however, equally clear that DEDS are fundamentally different from CVDS. They are event driven, asynchronous, mostly man-made and only became significant during the past generation. Increasingly, however, it can be argued that in the modem world our lives are being impacted by and dependent upon the efficient operations of such DEDS. Yet compared to the successful paradigm of differential equations for CVDS the mathematical modelling of DEDS is in its infancy. Nor are there as many successful and established techniques for their analysis and synthesis. The purpose of this series is to promote the study and understanding of the modelling, analysis, control, and management of DEDS. The idea of the series came from editing a special issue of the Proceedings of IEEE on DEOS during 1988.

Performance optimization is vital in the design and operation of modern engineering systems, including communications, manufacturing, robotics, and logistics. Most engineering systems are too complicated to model, or the system parameters cannot be easily identified, so learning techniques have to be applied. This is a multi-disciplinary area which has been attracting wide attention across many disciplines. Areas such as perturbation analysis (PA) in discrete event dynamic systems (DEDSs), Markov decision processes (MDPs) in operations research, reinforcement learning (RL) or neuro-dynamic programming (NDP) in computer science, identification and adaptive control (I&AC) in control systems, share the common goal: to make the "best decision" to optimize system performance.This book provides a unified framework based on a sensitivity point of view. It also introduces new approaches and proposes new research topics within this sensitivity-based framework.

Dynamic Systems (DEDS) are almost endless: military C31 Ilogistic systems, the emergency ward of a metropolitan hospital, back offices of large insurance and brokerage fums, service and spare part operations of multinational fums . . . . the point is the pervasive nature of such systems in the daily life of human beings. Yet DEDS is a relatively new phenomenon in dynamic systems studies. From the days of Galileo to Newton to quantum mechanics and cosmology of the present, dynamic systems in nature are primarily differential equations based and time driven. A large literature and endless success stories have been built up on such Continuous Variable Dynamic Systems (CVDS). It is, however, equally clear that DEDS are fundamentally different from CVDS. They are event driven, asynchronous, mostly man-made and only became significant during the past generation. Increasingly, however, it can be argued that in the modem world our lives are being impacted by and dependent upon the efficient operations of such DEDS. Yet compared to the successful paradigm of differential equations for CVDS the mathematical modelling of DEDS is in its infancy. Nor are there as many successful and established techniques for their analysis and synthesis. The purpose of this series is to promote the study and understanding of the modelling, analysis, control, and management of DEDS. The idea of the series came from editing a special issue of the Proceedings of IEEE on DEOS during 1988.

This monograph applies the relative optimization approach to time nonhomogeneous continuous-time and continuous-state dynamic systems. The approach is intuitively clear and does not require deep knowledge of the mathematics of partial differential equations. The topics covered have the following distinguishing features: long-run average with no under-selectivity, non-smooth value functions with no viscosity solutions, diffusion processes with degenerate points, multi-class optimization with state classification, and optimization with no dynamic programming.The book begins with an introduction to relative optimization, including a comparison with the traditional approach of dynamic programming. The text then studies the Markov process, focusing on infinite-horizon optimization problems, and moves on to discuss optimal control of diffusion processes with semi-smooth value functions and degenerate points, and optimization of multi-dimensional diffusion processes. The book concludes with a brief overview of performance derivative-based optimization.Among the more important novel considerations presented are:the extension of the Hamilton–Jacobi–Bellman optimality condition from smooth to semi-smooth value functions by derivation of explicit optimality conditions at semi-smooth points and application of this result to degenerate and reflected processes;proof of semi-smoothness of the value function at degenerate points;attention to the under-selectivity issue for the long-run average and bias optimality; discussion of state classification for time nonhomogeneous continuous processes and multi-class optimization; anddevelopment of the multi-dimensional Tanaka formula for semi-smooth functions and application of this formula to stochastic control of multi-dimensional systems with degenerate points.The book will be of interest to researchers and students in the field of stochastic control andperformance optimization alike.

This monograph applies the relative optimization approach to time nonhomogeneous continuous-time and continuous-state dynamic systems. The approach is intuitively clear and does not require deep knowledge of the mathematics of partial differential equations. The topics covered have the following distinguishing features: long-run average with no under-selectivity, non-smooth value functions with no viscosity solutions, diffusion processes with degenerate points, multi-class optimization with state classification, and optimization with no dynamic programming.The book begins with an introduction to relative optimization, including a comparison with the traditional approach of dynamic programming. The text then studies the Markov process, focusing on infinite-horizon optimization problems, and moves on to discuss optimal control of diffusion processes with semi-smooth value functions and degenerate points, and optimization of multi-dimensional diffusion processes. The book concludes with a brief overview of performance derivative-based optimization.Among the more important novel considerations presented are:the extension of the Hamilton–Jacobi–Bellman optimality condition from smooth to semi-smooth value functions by derivation of explicit optimality conditions at semi-smooth points and application of this result to degenerate and reflected processes;proof of semi-smoothness of the value function at degenerate points;attention to the under-selectivity issue for the long-run average and bias optimality; discussion of state classification for time nonhomogeneous continuous processes and multi-class optimization; anddevelopment of the multi-dimensional Tanaka formula for semi-smooth functions and application of this formula to stochastic control of multi-dimensional systems with degenerate points.The book will be of interest to researchers and students in the field of stochastic control andperformance optimization alike.

Using confluencity and relative optimization, the author classifies states as confluent or branching and shows how the under-selectivity issue of the long-run average can be easily addressed, multi-class optimization implemented, and Nth biases and Blackwell optimality conditions derived.