Cambridge Series in Statistical and Probabilistic Mathematics – serie

This book is an introduction to the field of asymptotic statistics. The treatment is both practical and mathematically rigorous. In addition to most of the standard topics of an asymptotics course, including likelihood inference, M-estimation, the theory of asymptotic efficiency, U-statistics, and rank procedures, the book also presents recent research topics such as semiparametric models, the bootstrap, and empirical processes and their applications. The topics are organized from the central idea of approximation by limit experiments, which gives the book one of its unifying themes. This entails mainly the local approximation of the classical i.i.d. set up with smooth parameters by location experiments involving a single, normally distributed observation. Thus, even the standard subjects of asymptotic statistics are presented in a novel way. Suitable as a graduate or Master's level statistics text, this book will also give researchers an overview of research in asymptotic statistics.

This lively introduction to measure-theoretic probability theory covers laws of large numbers, central limit theorems, random walks, martingales, Markov chains, ergodic theorems, and Brownian motion. Concentrating on results that are the most useful for applications, this comprehensive treatment is a rigorous graduate text and reference. Operating under the philosophy that the best way to learn probability is to see it in action, the book contains extended examples that apply the theory to concrete applications. This fifth edition contains a new chapter on multidimensional Brownian motion and its relationship to partial differential equations (PDEs), an advanced topic that is finding new applications. Setting the foundation for this expansion, Chapter 7 now features a proof of Itô's formula. Key exercises that previously were simply proofs left to the reader have been directly inserted into the text as lemmas. The new edition re-instates discussion about the central limit theorem for martingales and stationary sequences.

Rigorous probabilistic arguments, built on the foundation of measure theory introduced eighty years ago by Kolmogorov, have invaded many fields. Students of statistics, biostatistics, econometrics, finance, and other changing disciplines now find themselves needing to absorb theory beyond what they might have learned in the typical undergraduate, calculus-based probability course. This 2002 book grew from a one-semester course offered for many years to a mixed audience of graduate and undergraduate students who have not had the luxury of taking a course in measure theory. The core of the book covers the basic topics of independence, conditioning, martingales, convergence in distribution, and Fourier transforms. In addition there are numerous sections treating topics traditionally thought of as more advanced, such as coupling and the KMT strong approximation, option pricing via the equivalent martingale measure, and the isoperimetric inequality for Gaussian processes. The book is not just a presentation of mathematical theory, but is also a discussion of why that theory takes its current form. It will be a secure starting point for anyone who needs to invoke rigorous probabilistic arguments and understand what they mean.

This book describes the Bayesian approach to statistics at a level suitable for final year undergraduate and Masters students. It is unusual in presenting Bayesian statistics with a practical flavor and an emphasis on mainstream statistics, showing how to infer scientific, medical, and social conclusions from numerical data. The authors draw on many years of experience with practical and research programs and describe many statistical methods, not readily available elsewhere. A first chapter on Fisherian methods, together with a strong overall emphasis on likelihood, makes the text suitable for mainstream statistics courses whose instructors wish to follow mixed or comparative philosophies. The other chapters contain important sections relating to many areas of statistics such as the linear model, categorical data analysis, time series and forecasting, mixture models, survival analysis, Bayesian smoothing, and non-linear random effects models. The text includes a large number of practical examples, worked examples, and exercises. It will be essential reading for all statisticians, statistics students, and related interdisciplinary researchers.

The theory of empirical processes provides valuable tools for the development of asymptotic theory in (nonparametric) statistical models, and makes possible the unified treatment of a number of them. This book reveals the relation between the asymptotic behaviour of M-estimators and the complexity of parameter space. Virtually all results are proved using only elementary ideas developed within the book; there is minimal recourse to abstract theoretical results. To make the results concrete, a detailed treatment is presented for two important examples of M-estimation, namely maximum likelihood and least squares. The theory also covers estimation methods using penalties and sieves. Many illustrative examples are given, including the Grenander estimator, estimation of functions of bounded variation, smoothing splines, partially linear models, mixture models and image analysis. Graduate students and professionals in statistics as well as those with an interest in applications, to such areas as econometrics, medical statistics, etc., will welcome this treatment.

This book explains how computer software is designed to perform the tasks required for sophisticated statistical analysis. For statisticians, it examines the nitty-gritty computational problems behind statistical methods. For mathematicians and computer scientists, it looks at the application of mathematical tools to statistical problems. The first half of the book offers a basic background in numerical analysis that emphasizes issues important to statisticians. The next several chapters cover a broad array of statistical tools, such as maximum likelihood and nonlinear regression. The author also treats the application of numerical tools; numerical integration and random number generation are explained in a unified manner reflecting complementary views of Monte Carlo methods. Each chapter contains exercises that range from simple questions to research problems. Most of the examples are accompanied by demonstration and source code available from the author's website. New in this second edition are demonstrations coded in R, as well as new sections on linear programming and the Nelder-Mead search algorithm.

The theory of random graphs began in the late 1950s in several papers by Erdos and Renyi. In the late twentieth century, the notion of six degrees of separation, meaning that any two people on the planet can be connected by a short chain of people who know each other, inspired Strogatz and Watts to define the small world random graph in which each site is connected to k close neighbors, but also has long-range connections. At a similar time, it was observed in human social and sexual networks and on the Internet that the number of neighbors of an individual or computer has a power law distribution. This inspired Barabasi and Albert to define the preferential attachment model, which has these properties. These two papers have led to an explosion of research. The purpose of this book is to use a wide variety of mathematical argument to obtain insights into the properties of these graphs. A unique feature is the interest in the dynamics of process taking place on the graph in addition to their geometric properties, such as connectedness and diameter.

This book explains how computer software is designed to perform the tasks required for sophisticated statistical analysis. For statisticians, it examines the nitty-gritty computational problems behind statistical methods. For mathematicians and computer scientists, it looks at the application of mathematical tools to statistical problems. The first half of the book offers a basic background in numerical analysis that emphasizes issues important to statisticians. The next several chapters cover a broad array of statistical tools, such as maximum likelihood and nonlinear regression. The author also treats the application of numerical tools; numerical integration and random number generation are explained in a unified manner reflecting complementary views of Monte Carlo methods. Each chapter contains exercises that range from simple questions to research problems. Most of the examples are accompanied by demonstration and source code available from the author's website. New in this second edition are demonstrations coded in R, as well as new sections on linear programming and the Nelder-Mead search algorithm.

Bayesian nonparametrics works - theoretically, computationally. The theory provides highly flexible models whose complexity grows appropriately with the amount of data. Computational issues, though challenging, are no longer intractable. All that is needed is an entry point: this intelligent book is the perfect guide to what can seem a forbidding landscape. Tutorial chapters by Ghosal, Lijoi and Prünster, Teh and Jordan, and Dunson advance from theory, to basic models and hierarchical modeling, to applications and implementation, particularly in computer science and biostatistics. These are complemented by companion chapters by the editors and Griffin and Quintana, providing additional models, examining computational issues, identifying future growth areas, and giving links to related topics. This coherent text gives ready access both to underlying principles and to state-of-the-art practice. Specific examples are drawn from information retrieval, NLP, machine vision, computational biology, biostatistics, and bioinformatics.

Aimed at advanced undergraduate and graduate students in mathematics and related disciplines, this book presents the concepts and results underlying the Bayesian, frequentist and Fisherian approaches, with particular emphasis on the contrasts between them. Computational ideas are explained, as well as basic mathematical theory. Written in a lucid and informal style, this concise text provides both basic material on the main approaches to inference, as well as more advanced material on developments in statistical theory, including: material on Bayesian computation, such as MCMC, higher-order likelihood theory, predictive inference, bootstrap methods and conditional inference. It contains numerous extended examples of the application of formal inference techniques to real data, as well as historical commentary on the development of the subject. Throughout, the text concentrates on concepts, rather than mathematical detail, while maintaining appropriate levels of formality. Each chapter ends with a set of accessible problems.

Bootstrap methods are computer-intensive methods of statistical analysis, which use simulation to calculate standard errors, confidence intervals, and significance tests. The methods apply for any level of modelling, and so can be used for fully parametric, semiparametric, and completely nonparametric analysis. This 1997 book gives a broad and up-to-date coverage of bootstrap methods, with numerous applied examples, developed in a coherent way with the necessary theoretical basis. Applications include stratified data; finite populations; censored and missing data; linear, nonlinear, and smooth regression models; classification; time series and spatial problems. Special features of the book include: extensive discussion of significance tests and confidence intervals; material on various diagnostic methods; and methods for efficient computation, including improved Monte Carlo simulation. Each chapter includes both practical and theoretical exercises. S-Plus programs for implementing the methods described in the text are available from the supporting website.

This book describes the Bayesian approach to statistics at a level suitable for final year undergraduate and Masters students. It is unusual in presenting Bayesian statistics with a practical flavor and an emphasis on mainstream statistics, showing how to infer scientific, medical, and social conclusions from numerical data. The authors draw on many years of experience with practical and research programs and describe many statistical methods, not readily available elsewhere. A first chapter on Fisherian methods, together with a strong overall emphasis on likelihood, makes the text suitable for mainstream statistics courses whose instructors wish to follow mixed or comparative philosophies. The other chapters contain important sections relating to many areas of statistics such as the linear model, categorical data analysis, time series and forecasting, mixture models, survival analysis, Bayesian smoothing, and non-linear random effects models. The text includes a large number of practical examples, worked examples, and exercises. It will be essential reading for all statisticians, statistics students, and related interdisciplinary researchers.

Markov chains are central to the understanding of random processes. This is not only because they pervade the applications of random processes, but also because one can calculate explicitly many quantities of interest. This textbook, aimed at advanced undergraduate or MSc students with some background in basic probability theory, focuses on Markov chains and quickly develops a coherent and rigorous theory whilst showing also how actually to apply it. Both discrete-time and continuous-time chains are studied. A distinguishing feature is an introduction to more advanced topics such as martingales and potentials in the established context of Markov chains. There are applications to simulation, economics, optimal control, genetics, queues and many other topics, and exercises and examples drawn both from theory and practice. It will therefore be an ideal text either for elementary courses on random processes or those that are more oriented towards applications.

This book should be on the shelf of every practising statistician who designs experiments. Good design considers units and treatments first, and then allocates treatments to units. It does not choose from a menu of named designs. This approach requires a notation for units that does not depend on the treatments applied. Most structure on the set of observational units, or on the set of treatments, can be defined by factors. This book develops a coherent framework for thinking about factors and their relationships, including the use of Hasse diagrams. These are used to elucidate structure, calculate degrees of freedom and allocate treatment subspaces to appropriate strata. Based on a one-term course the author has taught since 1989, the book is ideal for advanced undergraduate and beginning graduate courses. Examples, exercises and discussion questions are drawn from a wide range of real applications: from drug development, to agriculture, to manufacturing.

This introduction to wavelet analysis 'from the ground level and up', and to wavelet-based statistical analysis of time series focuses on practical discrete time techniques, with detailed descriptions of the theory and algorithms needed to understand and implement the discrete wavelet transforms. Numerous examples illustrate the techniques on actual time series. The many embedded exercises - with complete solutions provided in the Appendix - allow readers to use the book for self-guided study. Additional exercises can be used in a classroom setting. A Web site offers access to the time series and wavelets used in the book, as well as information on accessing software in S-Plus and other languages. Students and researchers wishing to use wavelet methods to analyze time series will find this book essential.

Models and likelihood are the backbone of modern statistics. This 2003 book gives an integrated development of these topics that blends theory and practice, intended for advanced undergraduate and graduate students, researchers and practitioners. Its breadth is unrivaled, with sections on survival analysis, missing data, Markov chains, Markov random fields, point processes, graphical models, simulation and Markov chain Monte Carlo, estimating functions, asymptotic approximations, local likelihood and spline regressions as well as on more standard topics such as likelihood and linear and generalized linear models. Each chapter contains a wide range of problems and exercises. Practicals in the S language designed to build computing and data analysis skills, and a library of data sets to accompany the book, are available over the Web.

This eagerly awaited textbook covers everything the graduate student in probability wants to know about Brownian motion, as well as the latest research in the area. Starting with the construction of Brownian motion, the book then proceeds to sample path properties like continuity and nowhere differentiability. Notions of fractal dimension are introduced early and are used throughout the book to describe fine properties of Brownian paths. The relation of Brownian motion and random walk is explored from several viewpoints, including a development of the theory of Brownian local times from random walk embeddings. Stochastic integration is introduced as a tool and an accessible treatment of the potential theory of Brownian motion clears the path for an extensive treatment of intersections of Brownian paths. An investigation of exceptional points on the Brownian path and an appendix on SLE processes, by Oded Schramm and Wendelin Werner, lead directly to recent research themes.

Discover what you can do with R! Introducing the R system, covering standard regression methods, then tackling more advanced topics, this book guides users through the practical, powerful tools that the R system provides. The emphasis is on hands-on analysis, graphical display, and interpretation of data. The many worked examples, from real-world research, are accompanied by commentary on what is done and why. The companion website has code and datasets, allowing readers to reproduce all analyses, along with solutions to selected exercises and updates. Assuming basic statistical knowledge and some experience with data analysis (but not R), the book is ideal for research scientists, final-year undergraduate or graduate-level students of applied statistics, and practising statisticians. It is both for learning and for reference. This third edition expands upon topics such as Bayesian inference for regression, errors in variables, generalized linear mixed models, and random forests.

Models and likelihood are the backbone of modern statistics. This 2003 book gives an integrated development of these topics that blends theory and practice, intended for advanced undergraduate and graduate students, researchers and practitioners. Its breadth is unrivaled, with sections on survival analysis, missing data, Markov chains, Markov random fields, point processes, graphical models, simulation and Markov chain Monte Carlo, estimating functions, asymptotic approximations, local likelihood and spline regressions as well as on more standard topics such as likelihood and linear and generalized linear models. Each chapter contains a wide range of problems and exercises. Practicals in the S language designed to build computing and data analysis skills, and a library of data sets to accompany the book, are available over the Web.

Semiparametric regression is concerned with the flexible incorporation of non-linear functional relationships in regression analyses. Any application area that benefits from regression analysis can also benefit from semiparametric regression. Assuming only a basic familiarity with ordinary parametric regression, this user-friendly book explains the techniques and benefits of semiparametric regression in a concise and modular fashion. The authors make liberal use of graphics and examples plus case studies taken from environmental, financial, and other applications. They include practical advice on implementation and pointers to relevant software. The 2003 book is suitable as a textbook for students with little background in regression as well as a reference book for statistically oriented scientists such as biostatisticians, econometricians, quantitative social scientists, epidemiologists, with a good working knowledge of regression and the desire to begin using more flexible semiparametric models. Even experts on semiparametric regression should find something new here.

Semiparametric regression is concerned with the flexible incorporation of non-linear functional relationships in regression analyses. Any application area that benefits from regression analysis can also benefit from semiparametric regression. Assuming only a basic familiarity with ordinary parametric regression, this user-friendly book explains the techniques and benefits of semiparametric regression in a concise and modular fashion. The authors make liberal use of graphics and examples plus case studies taken from environmental, financial, and other applications. They include practical advice on implementation and pointers to relevant software. The 2003 book is suitable as a textbook for students with little background in regression as well as a reference book for statistically oriented scientists such as biostatisticians, econometricians, quantitative social scientists, epidemiologists, with a good working knowledge of regression and the desire to begin using more flexible semiparametric models. Even experts on semiparametric regression should find something new here.

Rigorous probabilistic arguments, built on the foundation of measure theory introduced eighty years ago by Kolmogorov, have invaded many fields. Students of statistics, biostatistics, econometrics, finance, and other changing disciplines now find themselves needing to absorb theory beyond what they might have learned in the typical undergraduate, calculus-based probability course. This 2002 book grew from a one-semester course offered for many years to a mixed audience of graduate and undergraduate students who have not had the luxury of taking a course in measure theory. The core of the book covers the basic topics of independence, conditioning, martingales, convergence in distribution, and Fourier transforms. In addition there are numerous sections treating topics traditionally thought of as more advanced, such as coupling and the KMT strong approximation, option pricing via the equivalent martingale measure, and the isoperimetric inequality for Gaussian processes. The book is not just a presentation of mathematical theory, but is also a discussion of why that theory takes its current form. It will be a secure starting point for anyone who needs to invoke rigorous probabilistic arguments and understand what they mean.

Many electronic and acoustic signals can be modelled as sums of sinusoids and noise. However, the amplitudes, phases and frequencies of the sinusoids are often unknown and must be estimated in order to characterise the periodicity or near-periodicity of a signal and consequently to identify its source. This book presents and analyses several practical techniques used for such estimation. The problem of tracking slow frequency changes over time of a very noisy sinusoid is also considered. Rigorous analyses are presented via asymptotic or large sample theory, together with physical insight. The book focuses on achieving extremely accurate estimates when the signal to noise ratio is low but the sample size is large. Each chapter begins with a detailed overview, and many applications are given. Matlab code for the estimation techniques is also included. The book will thus serve as an excellent introduction and reference for researchers analysing such signals.

This book was first published in 2004. Many observed phenomena, from the changing health of a patient to values on the stock market, are characterised by quantities that vary over time: stochastic processes are designed to study them. This book introduces practical methods of applying stochastic processes to an audience knowledgeable only in basic statistics. It covers almost all aspects of the subject and presents the theory in an easily accessible form that is highlighted by application to many examples. These examples arise from dozens of areas, from sociology through medicine to engineering. Complementing these are exercise sets making the book suited for introductory courses in stochastic processes. Software (available from www.cambridge.org) is provided for the freely available R system for the reader to apply to all the models presented.

The estimation of noisily observed states from a sequence of data has traditionally incorporated ideas from Hilbert spaces and calculus-based probability theory. As conditional expectation is the key concept, the correct setting for filtering theory is that of a probability space. Graduate engineers, mathematicians and those working in quantitative finance wishing to use filtering techniques will find in the first half of this book an accessible introduction to measure theory, stochastic calculus, and stochastic processes, with particular emphasis on martingales and Brownian motion. Exercises are included. The book then provides an excellent users' guide to filtering: basic theory is followed by a thorough treatment of Kalman filtering, including recent results which extend the Kalman filter to provide parameter estimates. These ideas are then applied to problems arising in finance, genetics and population modelling in three separate chapters, making this a comprehensive resource for both practitioners and researchers.

Aimed at advanced undergraduate and graduate students in mathematics and related disciplines, this book presents the concepts and results underlying the Bayesian, frequentist and Fisherian approaches, with particular emphasis on the contrasts between them. Computational ideas are explained, as well as basic mathematical theory. Written in a lucid and informal style, this concise text provides both basic material on the main approaches to inference, as well as more advanced material on developments in statistical theory, including: material on Bayesian computation, such as MCMC, higher-order likelihood theory, predictive inference, bootstrap methods and conditional inference. It contains numerous extended examples of the application of formal inference techniques to real data, as well as historical commentary on the development of the subject. Throughout, the text concentrates on concepts, rather than mathematical detail, while maintaining appropriate levels of formality. Each chapter ends with a set of accessible problems.

Experimental data can often be associated with or indexed by certain symmetrically interesting structures or sets of labels that appear, for example, in the study of short symbolic sequences in molecular biology, in preference or voting data, in (corneal) curvature data, and in studies of the handedness and entropy of symbolic sequences and elementary images. The symmetry studies introduced in this book describe the interplay among symmetry transformations that are characteristic of these sets of labels, their resulting classification, the algebraic decomposition of the data indexed by them, and the statistical analysis of the invariants induced by those decompositions. The overall purpose is to facilitate and guide the statistical study of the structured data from both a descriptive and inferential perspective. The text combines notions of algebra and statistics and develops a systematic methodology to better explore the interplay between symmetry-related research questions and their statistical analysis.

This detailed introduction to distribution theory uses no measure theory, making it suitable for students in statistics and econometrics as well as for researchers who use statistical methods. Good backgrounds in calculus and linear algebra are important and a course in elementary mathematical analysis is useful, but not required. An appendix gives a detailed summary of the mathematical definitions and results that are used in the book. Topics covered range from the basic distribution and density functions, expectation, conditioning, characteristic functions, cumulants, convergence in distribution and the central limit theorem to more advanced concepts such as exchangeability, models with a group structure, asymptotic approximations to integrals, orthogonal polynomials and saddlepoint approximations. The emphasis is on topics useful in understanding statistical methodology; thus, parametric statistical models and the distribution theory associated with the normal distribution are covered comprehensively.

In fields such as biology, medical sciences, sociology, and economics researchers often face the situation where the number of available observations, or the amount of available information, is sufficiently small that approximations based on the normal distribution may be unreliable. Theoretical work over the last quarter-century has led to new likelihood-based methods that lead to very accurate approximations in finite samples, but this work has had limited impact on statistical practice. This book illustrates by means of realistic examples and case studies how to use the new theory, and investigates how and when it makes a difference to the resulting inference. The treatment is oriented towards practice and comes with code in the R language (available from the web) which enables the methods to be applied in a range of situations of interest to practitioners. The analysis includes some comparisons of higher order likelihood inference with bootstrap or Bayesian methods.