Xun Yu Zhou – författare

The objective of this volume is to highlight through a collection of chap­ ters some of the recent research works in applied prob ability, specifically stochastic modeling and optimization. The volume is organized loosely into four parts. The first part is a col­ lection of several basic methodologies: singularly perturbed Markov chains (Chapter 1), and related applications in stochastic optimal control (Chapter 2); stochastic approximation, emphasizing convergence properties (Chapter 3); a performance-potential based approach to Markov decision program­ ming (Chapter 4); and interior-point techniques (homogeneous self-dual embedding and central path following) applied to stochastic programming (Chapter 5). The three chapters in the second part are concerned with queueing the­ ory. Chapters 6 and 7 both study processing networks - a general dass of queueing networks - focusing, respectively, on limit theorems in the form of strong approximation, and the issue of stability via connections to re­ lated fluid models. The subject of Chapter 8 is performance asymptotics via large deviations theory, when the input process to a queueing system exhibits long-range dependence, modeled as fractional Brownian motion.

In the mathematical treatment of many problems which arise in physics, economics, engineering, management, etc., the researcher frequently faces two major difficulties: infinite dimensionality and randomness of the evolution process. Infinite dimensionality occurs when the evolution in time of a process is accompanied by a space-like dependence; for example, spatial distribution of the temperature for a heat-conductor, spatial dependence of the time-varying displacement of a membrane subject to external forces, etc. Randomness is intrinsic to the mathematical formulation of many phenomena, such as fluctuation in the stock market, or noise in communication networks. Control theory of distributed parameter systems and stochastic systems focuses on physical phenomena which are governed by partial differential equations, delay-differential equations, integral differential equations, etc., and stochastic differential equations of various types. This has been a fertile field of research with over 40 years of history, which continues to be very active under the thrust of new emerging applications. Among the subjects covered are: Control of distributed parameter systems; Stochastic control; Applications in finance/insurance/manufacturing; Adapted control; Numerical approximation . It is essential reading for applied mathematicians, control theorists, economic/financial analysts and engineers.

The objective of this volume is to highlight through a collection of chap­ ters some of the recent research works in applied prob ability, specifically stochastic modeling and optimization. The volume is organized loosely into four parts. The first part is a col­ lection of several basic methodologies: singularly perturbed Markov chains (Chapter 1), and related applications in stochastic optimal control (Chapter 2); stochastic approximation, emphasizing convergence properties (Chapter 3); a performance-potential based approach to Markov decision program­ ming (Chapter 4); and interior-point techniques (homogeneous self-dual embedding and central path following) applied to stochastic programming (Chapter 5). The three chapters in the second part are concerned with queueing the­ ory. Chapters 6 and 7 both study processing networks - a general dass of queueing networks - focusing, respectively, on limit theorems in the form of strong approximation, and the issue of stability via connections to re­ lated fluid models. The subject of Chapter 8 is performance asymptotics via large deviations theory, when the input process to a queueing system exhibits long-range dependence, modeled as fractional Brownian motion.

As is well known, Pontryagin's maximum principle and Bellman's dynamic programming are the two principal and most commonly used approaches in solving stochastic optimal control problems. * An interesting phenomenon one can observe from the literature is that these two approaches have been developed separately and independently. Since both methods are used to investigate the same problems, a natural question one will ask is the fol­ lowing: (Q) What is the relationship betwccn the maximum principlc and dy­ namic programming in stochastic optimal controls? There did exist some researches (prior to the 1980s) on the relationship between these two. Nevertheless, the results usually werestated in heuristic terms and proved under rather restrictive assumptions, which were not satisfied in most cases. In the statement of a Pontryagin-type maximum principle there is an adjoint equation, which is an ordinary differential equation (ODE) in the (finite-dimensional) deterministic case and a stochastic differential equation (SDE) in the stochastic case. The system consisting of the adjoint equa­ tion, the original state equation, and the maximum condition is referred to as an (extended) Hamiltonian system. On the other hand, in Bellman's dynamic programming, there is a partial differential equation (PDE), of first order in the (finite-dimensional) deterministic case and of second or­ der in the stochastic case. This is known as a Hamilton-Jacobi-Bellman (HJB) equation.

In the mathematical treatment of many problems which arise in physics, economics, engineering, management, etc., the researcher frequently faces two major difficulties: infinite dimensionality and randomness of the evolution process. Infinite dimensionality occurs when the evolution in time of a process is accompanied by a space-like dependence; for example, spatial distribution of the temperature for a heat-conductor, spatial dependence of the time-varying displacement of a membrane subject to external forces, etc. Randomness is intrinsic to the mathematical formulation of many phenomena, such as fluctuation in the stock market, or noise in communication networks. Control theory of distributed parameter systems and stochastic systems focuses on physical phenomena which are governed by partial differential equations, delay-differential equations, integral differential equations, etc., and stochastic differential equations of various types. This has been a fertile field of research with over 40 years of history, which continues to be very active under the thrust of new emerging applications.Among the subjects covered are: - Control of distributed parameter systems; - Stochastic control; - Applications in finance/insurance/manufacturing; - Adapted control; - Numerical approximation. It is essential reading for applied mathematicians, control theorists, economic/financial analysts and engineers.

The objective of this volume is to highlight through a collection of chap­ ters some of the recent research works in applied prob ability, specifically stochastic modeling and optimization. The volume is organized loosely into four parts. The first part is a col­ lection of several basic methodologies: singularly perturbed Markov chains (Chapter 1), and related applications in stochastic optimal control (Chapter 2); stochastic approximation, emphasizing convergence properties (Chapter 3); a performance-potential based approach to Markov decision program­ ming (Chapter 4); and interior-point techniques (homogeneous self-dual embedding and central path following) applied to stochastic programming (Chapter 5). The three chapters in the second part are concerned with queueing the­ ory. Chapters 6 and 7 both study processing networks - a general dass of queueing networks - focusing, respectively, on limit theorems in the form of strong approximation, and the issue of stability via connections to re­ lated fluid models. The subject of Chapter 8 is performance asymptotics via large deviations theory, when the input process to a queueing system exhibits long-range dependence, modeled as fractional Brownian motion.

As is well known, Pontryagin''s maximum principle and Bellman''s dynamic programming are the two principal and most commonly used approaches in solving stochastic optimal control problems. * An interesting phenomenon one can observe from the literature is that these two approaches have been developed separately and independently. Since both methods are used to investigate the same problems, a natural question one will ask is the fol­ lowing: (Q) What is the relationship betwccn the maximum principlc and dy­ namic programming in stochastic optimal controls? There did exist some researches (prior to the 1980s) on the relationship between these two. Nevertheless, the results usually werestated in heuristic terms and proved under rather restrictive assumptions, which were not satisfied in most cases. In the statement of a Pontryagin-type maximum principle there is an adjoint equation, which is an ordinary differential equation (ODE) in the (finite-dimensional) deterministic case and a stochastic differential equation (SDE) in the stochastic case. The system consisting of the adjoint equa­ tion, the original state equation, and the maximum condition is referred to as an (extended) Hamiltonian system. On the other hand, in Bellman''s dynamic programming, there is a partial differential equation (PDE), of first order in the (finite-dimensional) deterministic case and of second or­ der in the stochastic case. This is known as a Hamilton-Jacobi-Bellman (HJB) equation.

As is well known, Pontryagin's maximum principle and Bellman's dynamic programming are the two principal and most commonly used approaches in solving stochastic optimal control problems. * An interesting phenomenon one can observe from the literature is that these two approaches have been developed separately and independently. Since both methods are used to investigate the same problems, a natural question one will ask is the fol­ lowing: (Q) What is the relationship betwccn the maximum principlc and dy­ namic programming in stochastic optimal controls? There did exist some researches (prior to the 1980s) on the relationship between these two. Nevertheless, the results usually werestated in heuristic terms and proved under rather restrictive assumptions, which were not satisfied in most cases. In the statement of a Pontryagin-type maximum principle there is an adjoint equation, which is an ordinary differential equation (ODE) in the (finite-dimensional) deterministic case and a stochastic differential equation (SDE) in the stochastic case. The system consisting of the adjoint equa­ tion, the original state equation, and the maximum condition is referred to as an (extended) Hamiltonian system. On the other hand, in Bellman's dynamic programming, there is a partial differential equation (PDE), of first order in the (finite-dimensional) deterministic case and of second or­ der in the stochastic case. This is known as a Hamilton-Jacobi-Bellman (HJB) equation.