Information Science and Statistics – serie

This book is a comprehensive and accessible introduction to the cross-entropy (CE) method. The CE method started life around 1997 when the first author proposed an adaptive algorithm for rare-event simulation using a cross-entropy minimization technique. It was soon realized that the underlying ideas had a much wider range of application than just in rare-event simulation; they could be readily adapted to tackle quite general combinatorial and multi-extremal optimization problems, including many problems associated with the field of learning algorithms and neural computation. The book is based on an advanced undergraduate course on the CE method, given at the Israel Institute of Technology (Technion) for the last three years. It is aimed at a broad audience of engineers, computer scientists, mathematicians, statisticians and in general anyone, theorist or practitioner, who is interested in smart simulation, fast optimization, learning algorithms, image processing, etc. Our aim was to write a book on the CE method which was accessible to advanced undergraduate students and engineers who simply want to apply the CE method in their work, while at the same time accentu­ ating the unifying and novel mathematical ideas behind the CE method, so as to stimulate further research at a postgraduate level.

The Minimum Message Length (MML) Principle is an information-theoretic approach to induction, hypothesis testing, model selection, and statistical inference. MML, which provides a formal specification for the implementation of Occam's Razor, asserts that the �best� explanation of observed data is the shortest. Further, an explanation is acceptable (i.e. the induction is justified) only if the explanation is shorter than the original data.This book gives a sound introduction to the Minimum Message Length Principle and its applications, provides the theoretical arguments for the adoption of the principle, and shows the development of certain approximations that assist its practical application. MML appears also to provide both a normative and a descriptive basis for inductive reasoning generally, and scientific induction in particular. The book describes this basis and aims to show its relevance to the Philosophy of Science.Statistical and Inductive Inference by Minimum Message Length will be of special interest to graduate students and researchers in Machine Learning and Data Mining, scientists and analysts in various disciplines wishing to make use of computer techniques for hypothesis discovery, statisticians and econometricians interested in the underlying theory of their discipline, and persons interested in the Philosophy of Science.The book could also be used in a graduate-level course in Machine Learning and Estimation and Model-selection, Econometrics and Data Mining.C.S. Wallace was appointed Foundation Chair of Computer Science at Monash University in 1968, at the age of 35, where he worked until his death in 2004. He received an ACM Fellowship in 1995, and was appointed Professor Emeritus in 1996. Professor Wallace made numerous significant contributions to diverse areas of Computer Science, such as Computer Architecture, Simulation and Machine Learning. His final research focused primarily on the Minimum Message Length Principle.

Twenty-?ve years have passed since the publication of the Russian version of the book Estimation of Dependencies Based on Empirical Data (EDBED for short).

Pattern recognition has its origins in engineering, whereas machine learning grew out of computer science. In particular, Bayesian methods have grown from a specialist niche to become mainstream, while graphical models have emerged as a general framework for describing and applying probabilistic models.

No statistical model is "true" or "false," "right" or "wrong"; the models just have varying performance, which can be assessed. The main theme in this book is to teach modeling based on the principle that the objective is to extract the information from data that can be learned with suggested classes of probability models. The intuitive and fundamental concepts of complexity, learnable information, and noise are formalized, which provides a firm information theoretic foundation for statistical modeling. Inspired by Kolmogorov's structure function in the algorithmic theory of complexity, this is accomplished by finding the shortest code length, called the stochastic complexity, with which the data can be encoded when advantage is taken of the models in a suggested class, which amounts to the MDL (Minimum Description Length) principle. The complexity, in turn, breaks up into the shortest code length for the optimal model in a set of models that can be optimally distinguished from the given data and the rest, which defines "noise" as the incompressible part in the data without useful information.Such a view of the modeling problem permits a unified treatment of any type of parameters, their number, and even their structure. Since only optimally distinguished models are worthy of testing, we get a logically sound and straightforward treatment of hypothesis testing, in which for the first time the confidence in the test result can be assessed. Although the prerequisites include only basic probability calculus and statistics, a moderate level of mathematical proficiency would be beneficial. The different and logically unassailable view of statistical modelling should provide excellent grounds for further research and suggest topics for graduate students in all fields of modern engineering, including and not restricted to signal and image processing, bioinformatics, pattern recognition, and machine learning to mention just a few.

However, since the late nineties, many new methods have been developed and nonlinear dimensionality reduction, also called manifold learning, has become a hot topic. The purpose of the book is to summarize clear facts and ideas about well-known methods as well as recent developments in the topic of nonlinear dimensionality reduction.

Probabilistic graphical models and decision graphs are powerful modeling tools for reasoning and decision making under uncertainty. As modeling languages they allow a natural specification of problem domains with inherent uncertainty, and from a computational perspective they support efficient algorithms for automatic construction and query answering. This includes belief updating, finding the most probable explanation for the observed evidence, detecting conflicts in the evidence entered into the network, determining optimal strategies, analyzing for relevance, and performing sensitivity analysis.The book introduces probabilistic graphical models and decision graphs, including Bayesian networks and influence diagrams. The reader is introduced to the two types of frameworks through examples and exercises, which also instruct the reader on how to build these models. The book is a new edition of Bayesian Networks and Decision Graphs by Finn V. Jensen. The new edition is structured into two parts. The first part focuses on probabilistic graphical models. Compared with the previous book, the new edition also includes a thorough description of recent extensions to the Bayesian network modeling language, advances in exact and approximate belief updating algorithms, and methods for learning both the structure and the parameters of a Bayesian network. The second part deals with decision graphs, and in addition to the frameworks described in the previous edition, it also introduces Markov decision processes and partially ordered decision problems. The authors also provide a well-founded practical introduction to Bayesian networks, object-oriented Bayesian networks, decision trees, influence diagrams (and variants hereof), and Markov decision processes.give practical advice on the construction of Bayesian networks, decision trees, and influence diagrams from domain knowledge.

WINNER OF THE 2001 DEGROOT PRIZE!Probabilistic expert systems are graphical networks that support the modelling of uncertainty and decisions in large complex domains, while retaining ease of calculation. Building on original research by the authors over a number of years, this book gives a thorough and rigorous mathematical treatment of the underlying ideas, structures, and algorithms, emphasizing those cases in which exact answers are obtainable. It covers both the updating of probabilistic uncertainty in the light of new evidence, and statistical inference, about unknown probabilities or unknown model structure, in the light of new data. The careful attention to detail will make this work an important reference source for all those involved in the theory and applications of probabilistic expert systems.This book was awarded the first DeGroot Prize by the International Society for Bayesian Analysis for a book making an important, timely, thorough, and notably original contribution to the statistics literature.

Every mathematical discipline goes through three periods of development: the naive, the formal, and the critical. David Hilbert The goal of this book is to explain the principles that made support vector machines (SVMs) a successful modeling and prediction tool for a variety of applications. We try to achieve this by presenting the basic ideas of SVMs together with the latest developments and current research questions in a uni?ed style. In a nutshell, we identify at least three reasons for the success of SVMs: their ability to learn well with only a very small number of free parameters, their robustness against several types of model violations and outliers, and last but not least their computational e?ciency compared with several other methods. Although there are several roots and precursors of SVMs, these methods gained particular momentum during the last 15 years since Vapnik (1995, 1998) published his well-known textbooks on statistical learning theory with aspecialemphasisonsupportvectormachines. Sincethen,the?eldofmachine learninghaswitnessedintenseactivityinthestudyofSVMs,whichhasspread moreandmoretootherdisciplinessuchasstatisticsandmathematics. Thusit seems fair to say that several communities are currently working on support vector machines and on related kernel-based methods. Although there are many interactions between these communities, we think that there is still roomforadditionalfruitfulinteractionandwouldbegladifthistextbookwere found helpful in stimulating further research. Many of the results presented in this book have previously been scattered in the journal literature or are still under review. As a consequence, these results have been accessible only to a relativelysmallnumberofspecialists,sometimesprobablyonlytopeoplefrom one community but not the others.

Monte Carlo methods are revolutionising the on-line analysis of data in fields as diverse as financial modelling, target tracking and computer vision. These methods, appearing under the names of bootstrap filters, condensation, optimal Monte Carlo filters, particle filters and survial of the fittest, have made it possible to solve numerically many complex, non-standarard problems that were previously intractable. This book presents the first comprehensive treatment of these techniques, including convergence results and applications to tracking, guidance, automated target recognition, aircraft navigation, robot navigation, econometrics, financial modelling, neural networks,optimal control, optimal filtering, communications, reinforcement learning, signal enhancement, model averaging and selection, computer vision, semiconductor design, population biology, dynamic Bayesian networks, and time series analysis. This will be of great value to students, researchers and practicioners, who have some basic knowledge of probability. Arnaud Doucet received the Ph. D. degree from the University of Paris- XI Orsay in 1997. From 1998 to 2000, he conducted research at the Signal Processing Group of Cambridge University, UK. He is currently an assistant professor at the Department of Electrical Engineering of Melbourne University, Australia. His research interests include Bayesian statistics, dynamic models and Monte Carlo methods. Nando de Freitas obtained a Ph.D. degree in information engineering from Cambridge University in 1999. He is presently a research associate with the artificial intelligence group of the University of California at Berkeley. His main research interests are in Bayesian statistics and the application of on-line and batch Monte Carlo methods to machine learning.

This book covers the basic statistical and analytical techniques of computer intrusion detection. It is aimed at both statisticians looking to become involved in the data analysis aspects of computer security and computer scientists looking to expand their toolbox of techniques for detecting intruders. The book is self-contained, assumng no expertise in either computer security or statistics. It begins with a description of the basics of TCP/IP, followed by chapters dealing with network traffic analysis, network monitoring for intrusion detection, host based intrusion detection, and computer viruses and other malicious code. Each section develops the necessary tools as needed. There is an extensive discussion of visualization as it relates to network data and intrusion detection. The book also contains a large bibliography covering the statistical, machine learning, and pattern recognition literature related to network monitoring and intrusion detection. David Marchette is a scientist at the Naval Surface Warfacre Center in Dalhgren, Virginia. He has worked at Navy labs for 15 years, doing research in pattern recognition, computational statistics, and image analysis.He has been a fellow by courtesy in the mathematical sciences department of the Johns Hopkins University since 2000. He has been working in conputer intrusion detection for several years, focusing on statistical methods for anomaly detection and visualization. Dr. Marchette received a Masters in Mathematics from the University of California, San Diego in 1982 and a Ph.D. in Computational Sciences and Informatics from George Mason University in 1996.

Cumulative sum (CUSUM) control charting is a valuable tool for detecting and diagnosing persistent shifts in series of readings. It is used in traditional statistical process control (SPC) settings such as manufacturing, but is also effective in settings as diverse as personnel management, econometrics, and conventional data analysis. It is an essential tool for the quality professional. This book covers CUSUMs from an application-oriented viewpoint, while also providing the essential theoretical underpinning. It is accessible to anyone with a basic statistical training, and is aimed at quality practitioners, teachers and students of quality methodologies, and people interested in analysis of time-ordered data. The text is supported by a Web site containing CUSUM software and data sets. Douglas M. Hawkins is Chair of the Department of Applied Statistics, University of Minnesota. He is a Fellow of the American Statistical Association, a Member of the International Statistical Institute and a Senior member of the American Society for Quality Control. His work on multivariate CUSUMs won him the Ellis R. Ott Award for the best paper on quality published in 1993.He has been Associate Editor of Technometrics and Journal of the American Statistical Association. David H. Olwell is Associate Professor in the Department of Mathematical Sciences at the United States Military Academy. He is a member of the American Statistical Association, the American Society for Quality Control, and the Military Operations Research Society, where his work on applications of CUSUMs to managing sexual harassment was nominated for the 1998 Barchi prize. He is Editor of Mathematica

This monograph provides a thorough and coherent introduction to the mathematical properties of feedforward neural networks and to the computationally intensive methodology that has enabled their highly successful application to complex problems of pattern classification, forecasting, regression, and nonlinear systems modeling. The reader is provided with the information needed to make practical use of the powerful modeling and design tool of feedforward neural networks, as well as presented with the background needed to make contributions to several research frontiers. This work is therefore of interest to those in electrical engineering, operations research, computer science, and statistics who would like to use nonlinear modeling of stochastic phenomena to treat problems of pattern classification, forecasting, signal processing, machine intelligence, and nonlinear regression. T. L. Fine is Professor of Electrical Engineering at Cornell University.

Winner of the 2002 DeGroot Prize. Awarded by the International Society for Bayesian Analysis to a book judged to represent an important, timely, thorough, and notably original contribution to the statistics literature.Probabilistic expert systems are graphical networks that support the modelling of uncertainty and decisions in large complex domains, while retaining ease of calculation. Building on original research by the authors over a number of years, this book gives a thorough and rigorous mathematical treatment of the underlying ideas, structures, and algorithms, emphasizing those cases in which exact answers are obtainable. It covers both the updating of probabilistic uncertainty in the light of new evidence, and statistical inference, about unknown probabilities or unknown model structure, in the light of new data.The book will be of interest to researchers and graduate students in artificial intelligence who desire an understanding of the mathematical and statistical basis of probabilistic expert systems, and to students and research workers in statistics wanting an introduction to this fascinating and rapidly developing field.The careful attention to detail will also make this work an important reference source for all those involved in the theory and applications of probabilistic expert systems.Robert G. Cowell is a Lecturer in the Faculty of Actuarial Science and Statistics of the Sir John Cass Business School, City of London. He has been working in the field of probabilistic expert systems for over a decade, and has published a number of research and tutorial articles in the area.A. Philip Dawid is Pearson Professor of Statistics at University College London. He has served as Editor of the Journal of the Royal Statistical Society (Series B) and of Biometrika, and as President of the International Society for Bayesian Analysis. He holds the Royal Statistical Society Guy Medal in Bronze and in Silver, and the G. W. Snedecor Award for the Best Publication in Biometry.Steffen L. Lauritzen is Professor of Mathematics and Statistics at the University of Aalborg. He has served as Editor of the Scandinavian Journal of Statistics. He holds the Royal Statistical Guy Medal in Silver and is an Honorary Fellow of the same society. He has, jointly with David J.Spiegelhalter, received the American Statistical Association's award for an "Outstanding Statistical Application."David J. Spiegelhalter is a Senior Scientist at the MRC Biostatistics Unit in the Cambridge University Institute of Public Health. He has published extensively on Bayesian methodology and applications, and holds the Royal Statistical Society Guy Medal in Bronze and in Silver.

The aim of this book is to discuss the fundamental ideas which lie behind the statistical theory of learning and generalization. It considers learning as a general problem of function estimation based on empirical data. Omitting proofs and technical details, the author concentrates on discussing the main results of learning theory and their connections to fundamental problems in statistics. These include: * the setting of learning problems based on the model of minimizing the risk functional from empirical data * a comprehensive analysis of the empirical risk minimization principle including necessary and sufficient conditions for its consistency * non-asymptotic bounds for the risk achieved using the empirical risk minimization principle * principles for controlling the generalization ability of learning machines using small sample sizes based on these bounds * the Support Vector methods that control the generalization ability when estimating function using small sample size. The second edition of the book contains three new chapters devoted to further development of the learning theory and SVM techniques.These include: * the theory of direct method of learning based on solving multidimensional integral equations for density, conditional probability, and conditional density estimation * a new inductive principle of learning. Written in a readable and concise style, the book is intended for statisticians, mathematicians, physicists, and computer scientists. Vladimir N. Vapnik is Technology Leader AT&T Labs-Research and Professor of London University. He is one of the founders of

This bookisan outgrowthoften yearsof researchatthe Universityof Florida Computational NeuroEngineering Laboratory (CNEL) in the general area of statistical signal processing and machine learning. One of the goals of writing the book is exactly to bridge the two ?elds that share so many common problems and techniques but are not yet e?ectively collaborating. Unlikeotherbooks thatcoverthe state ofthe artinagiven?eld,this book cuts across engineering (signal processing) and statistics (machine learning) withacommontheme:learningseenfromthepointofviewofinformationt- orywithanemphasisonRenyi'sde?nitionofinformation.Thebasicapproach is to utilize the information theory descriptors of entropy and divergence as nonparametric cost functions for the design of adaptive systems in unsup- vised or supervised training modes. Hence the title: Information-Theoretic Learning (ITL). In the course of these studies, we discovered that the main idea enabling a synergistic view as well as algorithmic implementations, does not involve the conventional central moments of the data (mean and covariance). Rather, the core concept is the ?-norm of the PDF, in part- ular its expected value (?= 2), which we call the information potential. This operator and related nonparametric estimators link information theory, optimization of adaptive systems, and reproducing kernel Hilbert spaces in a simple and unconventional way.

This book is a comprehensive and accessible introduction to the cross-entropy (CE) method. The CE method started life around 1997 when the first author proposed an adaptive algorithm for rare-event simulation using a cross-entropy minimization technique. It was soon realized that the underlying ideas had a much wider range of application than just in rare-event simulation; they could be readily adapted to tackle quite general combinatorial and multi-extremal optimization problems, including many problems associated with the field of learning algorithms and neural computation. The book is based on an advanced undergraduate course on the CE method, given at the Israel Institute of Technology (Technion) for the last three years. It is aimed at a broad audience of engineers, computer scientists, mathematicians, statisticians and in general anyone, theorist or practitioner, who is interested in smart simulation, fast optimization, learning algorithms, image processing, etc. Our aim was to write a book on the CE method which was accessible to advanced undergraduate students and engineers who simply want to apply the CE method in their work, while at the same time accentu­ ating the unifying and novel mathematical ideas behind the CE method, so as to stimulate further research at a postgraduate level.

The Minimum Message Length (MML) Principle is an information-theoretic approach to induction, hypothesis testing, model selection, and statistical inference. MML, which provides a formal specification for the implementation of Occam's Razor, asserts that the 'best' explanation of observed data is the shortest. Further, an explanation is acceptable (i.e. the induction is justified) only if the explanation is shorter than the original data. This book gives a sound introduction to the Minimum Message Length Principle and its applications, provides the theoretical arguments for the adoption of the principle, and shows the development of certain approximations that assist its practical application. MML appears also to provide both a normative and a descriptive basis for inductive reasoning generally, and scientific induction in particular. The book describes this basis and aims to show its relevance to the Philosophy of Science.Statistical and Inductive Inference by Minimum Message Length will be of special interest to graduate students and researchers in Machine Learning and Data Mining, scientists and analysts in various disciplines wishing to make use of computer techniques for hypothesis discovery, statisticians and econometricians interested in the underlying theory of their discipline, and persons interested in the Philosophy of Science. The book could also be used in a graduate-level course in Machine Learning and Estimation and Model-selection, Econometrics and Data Mining. "Any statistician interested in the foundations of the discipline, or the deeper philosophical issues of inference, will find this volume a rewarding read." Short Book Reviews of the International Statistical Institute, December 2005

Twenty-?ve years have passed since the publication of the Russian version of the book Estimation of Dependencies Based on Empirical Data (EDBED for short).

No statistical model is "true" or "false," "right" or "wrong"; the models just have varying performance, which can be assessed. The main theme in this book is to teach modeling based on the principle that the objective is to extract the information from data that can be learned with suggested classes of probability models. The intuitive and fundamental concepts of complexity, learnable information, and noise are formalized, which provides a firm information theoretic foundation for statistical modeling. Inspired by Kolmogorov's structure function in the algorithmic theory of complexity, this is accomplished by finding the shortest code length, called the stochastic complexity, with which the data can be encoded when advantage is taken of the models in a suggested class, which amounts to the MDL (Minimum Description Length) principle. The complexity, in turn, breaks up into the shortest code length for the optimal model in a set of models that can be optimally distinguished from the given data and the rest, which defines "noise" as the incompressible part in the data without useful information.Such a view of the modeling problem permits a unified treatment of any type of parameters, their number, and even their structure. Since only optimally distinguished models are worthy of testing, we get a logically sound and straightforward treatment of hypothesis testing, in which for the first time the confidence in the test result can be assessed. Although the prerequisites include only basic probability calculus and statistics, a moderate level of mathematical proficiency would be beneficial. The different and logically unassailable view of statistical modelling should provide excellent grounds for further research and suggest topics for graduate students in all fields of modern engineering, including and not restricted to signal and image processing, bioinformatics, pattern recognition, and machine learning to mention just a few.

However, since the late nineties, many new methods have been developed and nonlinear dimensionality reduction, also called manifold learning, has become a hot topic. The purpose of the book is to summarize clear facts and ideas about well-known methods as well as recent developments in the topic of nonlinear dimensionality reduction.

Probabilistic graphical models and decision graphs are powerful modeling tools for reasoning and decision making under uncertainty. As modeling languages they allow a natural specification of problem domains with inherent uncertainty, and from a computational perspective they support efficient algorithms for automatic construction and query answering. This includes belief updating, finding the most probable explanation for the observed evidence, detecting conflicts in the evidence entered into the network, determining optimal strategies, analyzing for relevance, and performing sensitivity analysis.The book introduces probabilistic graphical models and decision graphs, including Bayesian networks and influence diagrams. The reader is introduced to the two types of frameworks through examples and exercises, which also instruct the reader on how to build these models. The book is a new edition of Bayesian Networks and Decision Graphs by Finn V. Jensen. The new edition is structured into two parts. The first part focuses on probabilistic graphical models. Compared with the previous book, the new edition also includes a thorough description of recent extensions to the Bayesian network modeling language, advances in exact and approximate belief updating algorithms, and methods for learning both the structure and the parameters of a Bayesian network. The second part deals with decision graphs, and in addition to the frameworks described in the previous edition, it also introduces Markov decision processes and partially ordered decision problems. The authors also provide a well-founded practical introduction to Bayesian networks, object-oriented Bayesian networks, decision trees, influence diagrams (and variants hereof), and Markov decision processes.give practical advice on the construction of Bayesian networks, decision trees, and influence diagrams from domain knowledge.

Probabilistic networks, also known as Bayesian networks and influence diagrams, have become one of the most promising technologies in the area of applied artificial intelligence, offering intuitive, efficient, and reliable methods for diagnosis, prediction, decision making, classification, troubleshooting, and data mining under uncertainty. Bayesian Networks and Influence Diagrams: A Guide to Construction and Analysis provides a comprehensive guide for practitioners who wish to understand, construct, and analyze intelligent systems for decision support based on probabilistic networks. Intended primarily for practitioners, this book does not require sophisticated mathematical skills or deep understanding of the underlying theory and methods nor does it discuss alternative technologies for reasoning under uncertainty. The theory and methods presented are illustrated through more than 140 examples, and exercises are included for the reader to check his/her level of understanding. The techniques and methods presented for knowledge elicitation, model construction and verification, modeling techniques and tricks, learning models from data, and analyses of models have all been developed and refined on the basis of numerous courses that the authors have held for practitioners worldwide.

Monte Carlo methods are revolutionising the on-line analysis of data in fields as diverse as financial modelling, target tracking and computer vision. These methods, appearing under the names of bootstrap filters, condensation, optimal Monte Carlo filters, particle filters and survial of the fittest, have made it possible to solve numerically many complex, non-standarard problems that were previously intractable. This book presents the first comprehensive treatment of these techniques, including convergence results and applications to tracking, guidance, automated target recognition, aircraft navigation, robot navigation, econometrics, financial modelling, neural networks,optimal control, optimal filtering, communications, reinforcement learning, signal enhancement, model averaging and selection, computer vision, semiconductor design, population biology, dynamic Bayesian networks, and time series analysis. This will be of great value to students, researchers and practicioners, who have some basic knowledge of probability. Arnaud Doucet received the Ph. D. degree from the University of Paris- XI Orsay in 1997. From 1998 to 2000, he conducted research at the Signal Processing Group of Cambridge University, UK. He is currently an assistant professor at the Department of Electrical Engineering of Melbourne University, Australia. His research interests include Bayesian statistics, dynamic models and Monte Carlo methods. Nando de Freitas obtained a Ph.D. degree in information engineering from Cambridge University in 1999. He is presently a research associate with the artificial intelligence group of the University of California at Berkeley. His main research interests are in Bayesian statistics and the application of on-line and batch Monte Carlo methods to machine learning.

In the fall of 1999, I was asked to teach a course on computer intrusion detection for the Department of Mathematical Sciences of The Johns Hopkins University. That course was the genesis of this book. I had been working in the field for several years at the Naval Surface Warfare Center, in Dahlgren, Virginia, under the auspices of the SHADOW program, with some funding by the Office of Naval Research. In designing the class, I was concerned both with giving an overview of the basic problems in computer security, and with providing information that was of interest to a department of mathematicians. Thus, the focus of the course was to be more on methods for modeling and detecting intrusions rather than one on how to secure one's computer against intrusions. The first task was to find a book from which to teach. I was familiar with several books on the subject, but they were all at either a high level, focusing more on the political and policy aspects of the problem, or were written for security analysts, with little to interest a mathematician. I wanted to cover material that would appeal to the faculty members of the department, some of whom ended up sitting in on the course, as well as providing some interesting problems for students. None of the books on the market at the time had an adequate discussion of mathematical issues related to intrusion detection.

The aim of this book is to discuss the fundamental ideas which lie behind the statistical theory of learning and generalization. It considers learning as a general problem of function estimation based on empirical data. Omitting proofs and technical details, the author concentrates on discussing the main results of learning theory and their connections to fundamental problems in statistics. These include: * the setting of learning problems based on the model of minimizing the risk functional from empirical data * a comprehensive analysis of the empirical risk minimization principle including necessary and sufficient conditions for its consistency * non-asymptotic bounds for the risk achieved using the empirical risk minimization principle * principles for controlling the generalization ability of learning machines using small sample sizes based on these bounds * the Support Vector methods that control the generalization ability when estimating function using small sample size. The second edition of the book contains three new chapters devoted to further development of the learning theory and SVM techniques. These include: * the theory of direct method of learning based on solving multidimensional integral equations for density, conditional probability, and conditional density estimation * a new inductive principle of learning. Written in a readable and concise style, the book is intended for statisticians, mathematicians, physicists, and computer scientists. Vladimir N. Vapnik is Technology Leader AT&T Labs-Research and Professor of London University. He is one of the founders of

Examples abound throughout remote sensing (satellite data mapping, data assimilation, climate-change studies, land use), medical imaging (organ segmentation, anomaly detection), computer vision (image classification, segmentation), and other 2D/3D problems (biological imaging, porous media).

Biometrics, the science of using physical traits to identify individuals, is playing an increasing role in our security-conscious society and across the globe. Biometric authentication, or bioauthentication, systems are being used to secure everything from amusement parks to bank accounts to military installations. Yet developments in this field have not been matched by an equivalent improvement in the statistical methods for evaluating these systems.Compensating for this need, this unique text/reference provides a basic statistical methodology for practitioners and testers of bioauthentication devices, supplying a set of rigorous statistical methods for evaluating biometric authentication systems. This framework of methods can be extended and generalized for a wide range of applications and tests.This is the first single resource on statistical methods for estimation and comparison of the performance of biometric authentication systems. The book focuses on six common performance metrics: for each metric, statistical methods are derived for a single system that incorporates confidence intervals, hypothesis tests, sample size calculations, power calculations and prediction intervals. These methods are also extended to allow for the statistical comparison and evaluation of multiple systems for both independent and paired data.Topics and features:* Provides a statistical methodology for the most common biometric performance metrics: failure to enroll (FTE), failure to acquire (FTA), false non-match rate (FNMR), false match rate (FMR), and receiver operating characteristic (ROC) curves* Presents methods for the comparison of two or more biometric performance metrics* Introduces a new bootstrap methodology for FMR and ROC curve estimation* Supplies more than 120 examples, using publicly available biometric data where possible* Discusses the addition of prediction intervals to the bioauthentication statistical toolset* Describes sample-size and power calculations for FTE, FTA, FNMR and FMRResearchers, managers and decisions makers needing to compare biometric systems across a variety of metrics will find within this reference an invaluable set of statistical tools. Written for an upper-level undergraduate or master’s level audience with a quantitative background, readers are also expected to have an understanding of the topics in a typical undergraduate statistics course.Dr. Michael E. Schuckers is Associate Professor of Statistics at St. Lawrence University, Canton, NY, and a member of the Center for Identification Technology Research.

Cumulative sum (CUSUM) control charting is a valuable tool for detecting and diagnosing persistent shifts in series of readings. It is used in traditional statistical process control (SPC) settings such as manufacturing, but is also effective in settings as diverse as personnel management, econometrics, and conventional data analysis. It is an essential tool for the quality professional. This book covers CUSUMs from an application-oriented viewpoint, while also providing the essential theoretical underpinning. It is accessible to anyone with a basic statistical training, and is aimed at quality practitioners, teachers and students of quality methodologies, and people interested in analysis of time-ordered data. The text is supported by a Web site containing CUSUM software and data sets. Douglas M. Hawkins is Chair of the Department of Applied Statistics, University of Minnesota. He is a Fellow of the American Statistical Association, a Member of the International Statistical Institute and a Senior member of the American Society for Quality Control. His work on multivariate CUSUMs won him the Ellis R. Ott Award for the best paper on quality published in 1993. He has been Associate Editor of Technometrics and Journal of the American Statistical Association. David H. Olwell is Associate Professor in the Department of Mathematical Sciences at the United States Military Academy. He is a member of the American Statistical Association, the American Society for Quality Control, and the Military Operations Research Society, where his work on applications of CUSUMs to managing sexual harassment was nominated for the 1998 Barchi prize. He is Editor of Mathematica