E. B. Dynkin – författare

The theory of Markov Processes has become a powerful tool in partial differential equations and potential theory with important applications to physics. Professor Dynkin has made many profound contributions to the subject and in this volume are collected several of his most important expository and survey articles. The content of these articles has not been covered in any monograph as yet. This account is accessible to graduate students in mathematics and operations research and will be welcomed by all those interested in stochastic processes and their applications.

This book is devoted to the systematic exposition of the contemporary theory of controlled Markov processes with discrete time parameter or in another termi­ nology multistage Markovian decision processes. We discuss the applications of this theory to various concrete problems. Particular attention is paid to mathe­ matical models of economic planning, taking account of stochastic factors. The authors strove to construct the exposition in such a way that a reader interested in the applications can get through the book with a minimal mathe­ matical apparatus. On the other hand, a mathematician will find, in the appropriate chapters, a rigorous theory of general control models, based on advanced measure theory, analytic set theory, measurable selection theorems, and so forth. We have abstained from the manner of presentation of many mathematical monographs, in which one presents immediately the most general situation and only then discusses simpler special cases and examples. Wishing to separate out difficulties, we introduce new concepts and ideas in the simplest setting, where they already begin to work. Thus, before considering control problems on an infinite time interval, we investigate in detail the case of the finite interval. Here we first study in detail models with finite state and action spaces-a case not requiring a departure from the realm of elementary mathematics, and at the same time illustrating the most important principles of the theory.

The modem theory of Markov processes has its origins in the studies of A. A. MARKOV (1906-1907) on sequences of experiments "connected in a chain" and in the attempts to describe mathematically the physical phenomenon known as Brownian motion (L. BACHELlER 1900, A. EIN- STEIN 1905). The first correct mathematical construction of a Markov process with continuous trajectories was given by N. WIENER in 1923. (This process is often called the Wiener process.) The general theory of Markov processes was developed in the 1930's and 1940's by A. N. KOL- MOGOROV, W. FELLER, W. DOEBLlN, P. LEVY, J. L. DOOB, and others. During the past ten years the theory of Markov processes has entered a new period of intensive development. The methods of the theory of semigroups of linear operators made possible further progress in the classification of Markov processes by their infinitesimal characteristics. The broad classes of Markov processes with continuous trajectories be- came the main object of study. The connections between Markov pro- cesses and classical analysis were further developed.It has become possible not only to apply the results and methods of analysis to the problems of probability theory, but also to investigate analytic problems using probabilistic methods. Remarkable new connections between Markov processes and potential theory were revealed. The foundations of the theory were reviewed critically: the new concept of strong Markov process acquired for the whole theory of Markov processes great importance.