Springer Texts in Electrical Engineering – serie

For most people, intuitive notions concerning probabilities are connected with relative frequencies of occurrence. For example, when we say that in toss- ing a coin, the probability of its coming up "heads" is 1/2, we usually mean that in a large number of tosses, about 1/2 of the tosses will come up heads. Unfortunately, relative frequency of occurrence has proved to be an unsatis- factory starting point in defining probability. Although there have been attempts to make frequency of occurrence part of the axiomatic structure of probability theory, the currently accepted formu1ation is one based on measure theory due to Ko1mogorov. In this formulation frequency of occurrence is an interpretation for probability rather than adefinition. This inter- pretation is justified under suitab1e conditions by the 1aw of 1arge numbers. The starting point of probability theory is usua11y taken to be an experi- ment the outcome of which is not fixed apriori. Some fami1iar examples inc1ude tossing a die, observation of a noise vo1tage at a fixed time, the error in measuring a physica1 parameter, and the exact touchdown time of an aircraft.Let ~ denote the set of all possib1e outcomes of an experiment. For examp1e, for the experiment of tossing one die, ~ = {1, 2, 3, 4, 5, 6}, whi1e for the touchdown time of an aircraft, ~ might be chosen to be 0 ~ t < 00. We note that for a given experiment on1y one outcome is ever observed.

This volume is the result of our teaching in the last few years of a first­ year graduate course on multivariable feedback systems addressed to control engineers. The prerequisites are modest: an undergraduate course in control (for acquaintance with concepts, terms, and design goals) and a senior-graduate course in linear systems. This volume covers lumped linear time-invariant multi-input multi-output systems with strong emphasis on control problems. The purpose is to provide a rapid introduction to some of the main and simpler results of control theory and to provide access to the current literature. Note that our exposition pays particular attention to the time-domain behavior of the systems under study. Note also that we cover neither optimization nor stochastic systems since these topics are treated in separate courses. As is obvious from its abundant literature, multivariable control is a very rapidly developing field. Consequently, we have no expectation that our exposition will become definitive; however, we hope that our efforts will be found useful. To get an idea of the contents, we suggest reading carefully the table of contents and the introduction of the chapters. Roughly, Chapter 1 is an introduction to feedback issues in a multivariable context (desensitization, large gain, singular values, etc. ). Chapters 2 and 3 cover the mathematical tools for handling transfer functions as polynomial-matrix fractions and for studying systems described by polynomial matrices. Chapter 4 uses these tools to cover the general theory of interconnected systems.

The purpose of this book is to introduce the reader to the basic theory of signal detection and estimation. It is assumed that the reader has a working knowledge of applied probability and random processes such as that taught in a typical first-semester graduate engineering course on these subjects. This material is covered, for example, in the book by Wong (1983) in this series. More advanced concepts in these areas are introduced where needed, primarily in Chapters VI and VII, where continuous-time problems are treated. This book is adapted from a one-semester, second-tier graduate course taught at the University of Illinois and at Princeton University. However, this material can also be used for a shorter or first-tier course by restricting coverage to Chapters I through V, which for the most part can be read with a background of only the basics of applied probability, including random vectors and conditional expectations. Sufficient background for the latter option is given for example in the book by Thomas (1986), also in this series. This treatment is also suitable for use as a text in other modes. For example, two smaller courses, one in signal detection (Chapters II, III, and VI) and one in estimation (Chapters IV, V, and VII), can be taught from the materials as organized here. Similarly, an introductory-level course (Chapters I through IV) followed by a more advanced course (Chapters V through VII) is another possibility.

This book was written for an introductory one-term course in probability. It is intended to provide the minimum background in probability that is necessary for students interested in applications to engineering and the sciences. Although it is aimed primarily at upperclassmen and beginning graduate students, the only prere­ quisite is the standard calculus course usually required of under­ graduates in engineering and science. Most beginning students will have some intuitive notions of the meaning of probability based on experiences involving, for example, games of chance. This book develops from these notions a set of precise and ordered concepts comprising the elementary theory of probability. An attempt has been made to state theorems carefully, but the level of the proofs varies greatly from formal arguments to appeals to intuition. The book is in no way intended as a substi­ tu te for a rigorous mathematical treatment of probability. How­ ever, some small amount of the language of formal mathematics is used, so that the student may become better prepared (at least psychologically) either for more formal courses or for study of the literature. Numerous examples are provided throughout the book. Many of these are of an elementary nature and are intended merely to illustrate textual material. A reasonable number of problems of varying difficulty are provided. Instructors who adopt the text for classroom use may obtain a Solutions Manual for all of the problems by writing to the author.

This book was written as a first treatment of statistical com­ munication theory and communication systems at a senior­ graduate level. The only formal prerequisite is a knowledge of ele­ mentary calculus; however, some familiarity with linear systems and transform theory will be helpful. Chapter 1 is introductory and contains no substantial techni­ cal material. Chapter 2 is an elementary introduction to probability theory at a nonrigorous and non abstract level. It is essential to the remainder of the book but may be skipped (or reviewed has­ tily) by any student who has taken a one-semester undergraduate course in probability. Chapter 3 is a brief treatment of random processes and spec­ tral analysis. It includes an introduction to shot noise (Sections 3.14-3.17) which is not subsequently used explicitly. Chapter 4 considers linear systems with random inputs. It includes a considerable amount of material on narrow-band sys­ tems and on the representation of random processes. Chapter5 treats the matched filter and the linear least mean-squared-error filter at an elementary level but in some detail. Numerous examples are provided throughout the book. Many of these are of an elementary nature and are intended merely to illustrate textual material. A reasonable number of problems of varying difficulty are provided. Instructors who adopt the text for classroom use may obtain a Solutions Manual for most of the problems by writing to the author.

This book is the result of our teaching over the years an undergraduate course on Linear Optimal Systems to applied mathematicians and a first-year graduate course on Linear Systems to engineers. The contents of the book bear the strong influence of the great advances in the field and of its enormous literature. However, we made no attempt to have a complete coverage. Our motivation was to write a book on linear systems that covers finite­ dimensional linear systems, always keeping in mind the main purpose of engineering and applied science, which is to analyze, design, and improve the performance of phy­ sical systems. Hence we discuss the effect of small nonlinearities, and of perturbations of feedback. It is our on the data; we face robustness issues and discuss the properties hope that the book will be a useful reference for a first-year graduate student. We assume that a typical reader with an engineering background will have gone through the conventional undergraduate single-input single-output linear systems course; an elementary course in control is not indispensable but may be useful for motivation. For readers from a mathematical curriculum we require only familiarity with techniques of linear algebra and of ordinary differential equations.

The purpose of this book is to introduce the reader to the basic theory of signal detection and estimation. It is assumed that the reader has a working knowledge of applied probability and random processes such as that taught in a typical first-semester graduate engineering course on these subjects. This material is covered, for example, in the book by Wong (1983) in this series. More advanced concepts in these areas are introduced where needed, primarily in Chapters VI and VII, where continuous-time problems are treated. This book is adapted from a one-semester, second-tier graduate course taught at the University of Illinois and at Princeton University. However, this material can also be used for a shorter or first-tier course by restricting coverage to Chapters I through V, which for the most part can be read with a background of only the basics of applied probability, including random vectors and conditional expectations. Sufficient background for the latter option is given for example in the book by Thomas (1986), also in this series. This treatment is also suitable for use as a text in other modes. For example, two smaller courses, one in signal detection (Chapters II, III, and VI) and one in estimation (Chapters IV, V, and VII), can be taught from the materials as organized here. Similarly, an introductory-level course (Chapters I through IV) followed by a more advanced course (Chapters V through VII) is another possibility.

This book is the result of our teaching over the years an undergraduate course on Linear Optimal Systems to applied mathematicians and a first-year graduate course on Linear Systems to engineers. The contents of the book bear the strong influence of the great advances in the field and of its enormous literature. However, we made no attempt to have a complete coverage. Our motivation was to write a book on linear systems that covers finite­ dimensional linear systems, always keeping in mind the main purpose of engineering and applied science, which is to analyze, design, and improve the performance of phy­ sical systems. Hence we discuss the effect of small nonlinearities, and of perturbations of feedback. It is our on the data; we face robustness issues and discuss the properties hope that the book will be a useful reference for a first-year graduate student. We assume that a typical reader with an engineering background will have gone through the conventional undergraduate single-input single-output linear systems course; an elementary course in control is not indispensable but may be useful for motivation. For readers from a mathematical curriculum we require only familiarity with techniques of linear algebra and of ordinary differential equations.

This book is a revision of Random Point Processes written by D. L. Snyder and published by John Wiley and Sons in 1975. More emphasis is given to point processes on multidimensional spaces, especially to pro­ cesses in two dimensions. This reflects the tremendous increase that has taken place in the use of point-process models for the description of data from which images of objects of interest are formed in a wide variety of scientific and engineering disciplines. A new chapter, Translated Poisson Processes, has been added, and several of the chapters of the fIrst edition have been modifIed to accommodate this new material. Some parts of the fIrst edition have been deleted to make room. Chapter 7 of the fIrst edition, which was about general marked point-processes, has been eliminated, but much of the material appears elsewhere in the new text. With some re­ luctance, we concluded it necessary to eliminate the topic of hypothesis testing for point-process models. Much of the material of the fIrst edition was motivated by the use of point-process models in applications at the Biomedical Computer Labo­ ratory of Washington University, as is evident from the following excerpt from the Preface to the first edition. "It was Jerome R. Cox, Jr. , founder and [1974] director of Washington University's Biomedical Computer Laboratory, who ftrst interested me [D. L. S.

This book contains a unified treatment of a class of problems of signal detection theory. This is the detection of signals in addi­ tive noise which is not required to have Gaussian probability den­ sity functions in its statistical description. For the most part the material developed here can be classified as belonging to the gen­ eral body of results of parametric theory. Thus the probability density functions of the observations are assumed to be known, at least to within a finite number of unknown parameters in a known functional form. Of course the focus is on noise which is not Gaussian; results for Gaussian noise in the problems treated here become special cases. The contents also form a bridge between the classical results of signal detection in Gaussian noise and those of nonparametric and robust signal detection, which are not con­ sidered in this book. Three canonical problems of signal detection in additive noise are covered here. These allow between them formulation of a range of specific detection problems arising in applications such as radar and sonar, binary signaling, and pattern recognition and classification. The simplest to state and perhaps the most widely studied of all is the problem of detecting a completely known deterministic signal in noise. Also considered here is the detection random non-deterministic signal in noise. Both of these situa­ of a tions may arise for observation processes of the low-pass type and also for processes of the band-pass type.

Due to a steady flow of requests over several years, Springer-Verlag now provides a corrected reprint of this text. It is designed to serve as a text for a first semester graduate level course for students in digital communication systems. As a pre­ requisite, it is presumed that the reader has an understanding of basic probability and stochastic processes. The treatment of digital communications in this book is intended to serve as an introduction to the subject. Part one is a development of the elements of statistical communication theory and radar detection. The text begins with a general model of a communication system which is extensively developed and the performance analyses of various conventional systems. The first part also serves as introductory material for the second part of the text which is a comprehensive study of the theory of transmitter optimization for coherent and noncoherent digital commu­ nication systems, that is, the theory of signal design.

This book is a revision of Stochastic Processes in Information and Dynamical Systems written by the first author (E.W.) and published in 1971. The book was originally written, and revised, to provide a graduate level text in stochastic processes for students whose primary interest is its applications. It treats both the traditional topic of sta­ tionary processes in linear time-invariant systems as well as the more modern theory of stochastic systems in which dynamic structure plays a profound role. Our aim is to provide a high-level, yet readily acces­ sible, treatment of those topics in the theory of continuous-parameter stochastic processes that are important in the analysis of information and dynamical systems. The theory of stochastic processes can easily become abstract. In dealing with it from an applied point of view, we have found it difficult to decide on the appropriate level of rigor. We intend to provide just enough mathematical machinery so that important results can be stated PREFACE vi with precision and clarity; so much ofthe theory of stochastic processes is inherently simple if the suitable framework is provided. The price of providing this framework seems worth paying even though the ul­ timate goal is in applications and not the mathematics per se.

Much of that which is ordinal is modeled as analog. Most computational engines on the other hand are dig- ital. Transforming from analog to digital is straightforward: we simply sample. Regaining the original signal from these samples or assessing the information lost in the sampling process are the fundamental questions addressed by sampling and interpolation theory. This book deals with understanding, generalizing, and extending the cardinal series of Shannon sampling theory. The fundamental form of this series states, remarkably, that a bandlimited signal is uniquely specified by its sufficiently close equally spaced samples. The contents of this book evolved from a set of lecture notes prepared for a graduate survey course on Shannon sampling and interpolation theory. The course was taught at the Department of Electrical Engineering at the University of Washington, Seattle. Each of the seven chapters in this book includes a list of references specific to that chapter. A sequel to this book will contain an extensive bibliography on the subject. The author has also opted to include solutions to selected exercises in the Appendix.

Advanced Topics in Shannon Sampling and Interpolation Theory is the second volume of a textbook on signal analysis solely devoted to the topic of sampling and restoration of continuous time signals and images. Sampling and reconstruction are fundamental problems in any field that deals with real-time signals or images, including communication engineering, image processing, seismology, speech recognition, and digital signal processing. This second volume includes contributions from leading researchers in the field on such topics as Gabor's signal expansion, sampling in optical image formation, linear prediction theory, polar and spiral sampling theory, interpolation from nonuniform samples, an extension of Papoulis's generalized sampling expansion to higher dimensions, and applications of sampling theory to optics and to time-frequency representations. The exhaustive bibliography on Shannon sampling theory will make this an invaluable research tool as well as an excellent text for students planning further research in the field.