Zhengyou Zhang – författare

Human faces are familiar to our visual systems. We easily recognize a person's face in arbitrary lighting conditions and in a variety of poses; detect small appearance changes; and notice subtle expression details. Can computer vision systems process face images as well as human vision systems can? Face image processing has potential applications in surveillance, image and video search, social networking and other domains. A comprehensive guide to this fascinating topic, this book provides a systematic description of modeling face geometry and appearance from images, including information on mathematical tools, physical concepts, image processing and computer vision techniques, and concrete prototype systems. The book will be an excellent reference for researchers and graduate students in computer vision, computer graphics and multimedia, as well as application developers who would like to gain a better understanding of the state of the art.

Appendix 164 3. A 3. A. 1 Approximate Estimation of Fundamental Matrix from General Matrix 164 3. A. 2 Estimation of Affine Transformation 165 4 RECOVERY OF EPIPOLAR GEOMETRY FROM LINE SEGMENTS OR LINES 167 Line Segments or Straight Lines 168 4. 1 4. 2 Solving Motion Using Line Segments Between Two Views 173 4. 2. 1 Overlap of Two Corresponding Line Segments 173 Estimating Motion by Maximizing Overlap 175 4. 2. 2 Implementation Details 4. 2. 3 176 Reconstructing 3D Line Segments 4. 2. 4 179 4. 2. 5 Experimental Results 180 4. 2. 6 Discussions 192 4. 3 Determining Epipolar Geometry of Three Views 194 4. 3. 1 Trifocal Constraints for Point Matches 194 4. 3. 2 Trifocal Constraints for Line Correspondences 199 4. 3. 3 Linear Estimation of K, L, and M Using Points and Lines 200 4. 3. 4 Determining Camera Projection Matrices 201 4. 3. 5 Image Transfer 203 4. 4 Summary 204 5 REDEFINING STEREO, MOTION AND OBJECT RECOGNITION VIA EPIPOLAR GEOMETRY 205 5. 1 Conventional Approaches to Stereo, Motion and Object Recognition 205 5. 1. 1 Stereo 205 5. 1. 2 Motion 206 5. 1. 3 Object Recognition 207 5. 2 Correspondence in Stereo, Motion and Object Recognition as 1D Search 209 5. 2. 1 Stereo Matching 209 xi Contents 5. 2. 2 Motion Correspondence and Segmentation 209 5. 2. 3 3D Object Recognition and Localization 210 Disparity and Spatial Disparity Space 210 5.

Face detection, because of its vast array of applications, is one of the most active research areas in computer vision. In this book, we review various approaches to face detection developed in the past decade, with more emphasis on boosting-based learning algorithms. We then present a series of algorithms that are empowered by the statistical view of boosting and the concept of multiple instance learning. We start by describing a boosting learning framework that is capable to handle billions of training examples. It differs from traditional bootstrapping schemes in that no intermediate thresholds need to be set during training, yet the total number of negative examples used for feature selection remains constant and focused (on the poor performing ones). A multiple instance pruning scheme is then adopted to set the intermediate thresholds after boosting learning. This algorithm generates detectors that are both fast and accurate. We then present two multiple instance learning schemesfor face detection, multiple instance learning boosting (MILBoost) and winner-take-all multiple category boosting (WTA-McBoost). MILBoost addresses the uncertainty in accurately pinpointing the location of the object being detected, while WTA-McBoost addresses the uncertainty in determining the most appropriate subcategory label for multiview object detection. Both schemes can resolve the ambiguity of the labeling process and reduce outliers during training, which leads to improved detector performances. In many applications, a detector trained with generic data sets may not perform optimally in a new environment. We propose detection adaption, which is a promising solution for this problem. We present an adaptation scheme based on the Taylor expansion of the boosting learning objective function, and we propose to store the second order statistics of the generic training data for future adaptation. We show that with a small amount of labeled data in the new environment, the detector'sperformance can be greatly improved. We also present two interesting applications where boosting learning was applied successfully. The first application is face verification for filtering and ranking image/video search results on celebrities. We present boosted multi-task learning (MTL), yet another boosting learning algorithm that extends MILBoost with a graphical model. Since the available number of training images for each celebrity may be limited, learning individual classifiers for each person may cause overfitting. MTL jointly learns classifiers for multiple people by sharing a few boosting classifiers in order to avoid overfitting. The second application addresses the need of speaker detection in conference rooms. The goal is to find who is speaking, given a microphone array and a panoramic video of the room. We show that by combining audio and visual features in a boosting framework, we can determine the speaker's position very accurately. Finally, we offer our thoughts on future directions for face detection. Table of Contents: A Brief Survey of the Face Detection Literature / Cascade-based Real-Time Face Detection / Multiple Instance Learning for Face Detection / Detector Adaptation / Other Applications / Conclusions and Future Work

Face detection, because of its vast array of applications, is one of the most active research areas in computer vision. In this book, we review various approaches to face detection developed in the past decade, with more emphasis on boosting-based learning algorithms. We then present a series of algorithms that are empowered by the statistical view of boosting and the concept of multiple instance learning. We start by describing a boosting learning framework that is capable to handle billions of training examples. It differs from traditional bootstrapping schemes in that no intermediate thresholds need to be set during training, yet the total number of negative examples used for feature selection remains constant and focused (on the poor performing ones). A multiple instance pruning scheme is then adopted to set the intermediate thresholds after boosting learning. This algorithm generates detectors that are both fast and accurate. We then present two multiple instance learning schemesfor face detection, multiple instance learning boosting (MILBoost) and winner-take-all multiple category boosting (WTA-McBoost). MILBoost addresses the uncertainty in accurately pinpointing the location of the object being detected, while WTA-McBoost addresses the uncertainty in determining the most appropriate subcategory label for multiview object detection. Both schemes can resolve the ambiguity of the labeling process and reduce outliers during training, which leads to improved detector performances. In many applications, a detector trained with generic data sets may not perform optimally in a new environment. We propose detection adaption, which is a promising solution for this problem. We present an adaptation scheme based on the Taylor expansion of the boosting learning objective function, and we propose to store the second order statistics of the generic training data for future adaptation. We show that with a small amount of labeled data in the new environment, the detector''sperformance can be greatly improved. We also present two interesting applications where boosting learning was applied successfully. The first application is face verification for filtering and ranking image/video search results on celebrities. We present boosted multi-task learning (MTL), yet another boosting learning algorithm that extends MILBoost with a graphical model. Since the available number of training images for each celebrity may be limited, learning individual classifiers for each person may cause overfitting. MTL jointly learns classifiers for multiple people by sharing a few boosting classifiers in order to avoid overfitting. The second application addresses the need of speaker detection in conference rooms. The goal is to find who is speaking, given a microphone array and a panoramic video of the room. We show that by combining audio and visual features in a boosting framework, we can determine the speaker''s position very accurately. Finally, we offer our thoughts on future directions for face detection. Table of Contents: A Brief Survey of the Face Detection Literature / Cascade-based Real-Time Face Detection / Multiple Instance Learning for Face Detection / Detector Adaptation / Other Applications / Conclusions and Future Work

This book presents an interdisciplinary selection of cutting-edge research on RGB-D based computer vision. Features: discusses the calibration of color and depth cameras, the reduction of noise on depth maps and methods for capturing human performance in 3D; reviews a selection of applications which use RGB-D information to reconstruct human figures, evaluate energy consumption and obtain accurate action classification; presents an approach for 3D object retrieval and for the reconstruction of gas flow from multiple Kinect cameras; describes an RGB-D computer vision system designed to assist the visually impaired and another for smart-environment sensing to assist elderly and disabled people; examines the effective features that characterize static hand poses and introduces a unified framework to enforce both temporal and spatial constraints for hand parsing; proposes a new classifier architecture for real-time hand pose recognition and a novel hand segmentation and gesture recognition system.

This book presents an interdisciplinary selection of cutting-edge research on RGB-D based computer vision. Features: discusses the calibration of color and depth cameras, the reduction of noise on depth maps and methods for capturing human performance in 3D; reviews a selection of applications which use RGB-D information to reconstruct human figures, evaluate energy consumption and obtain accurate action classification; presents an approach for 3D object retrieval and for the reconstruction of gas flow from multiple Kinect cameras; describes an RGB-D computer vision system designed to assist the visually impaired and another for smart-environment sensing to assist elderly and disabled people; examines the effective features that characterize static hand poses and introduces a unified framework to enforce both temporal and spatial constraints for hand parsing; proposes a new classifier architecture for real-time hand pose recognition and a novel hand segmentation and gesture recognition system.

This book presents an interdisciplinary selection of cutting-edge research on RGB-D based computer vision. Features: discusses the calibration of color and depth cameras, the reduction of noise on depth maps and methods for capturing human performance in 3D; reviews a selection of applications which use RGB-D information to reconstruct human figures, evaluate energy consumption and obtain accurate action classification; presents an approach for 3D object retrieval and for the reconstruction of gas flow from multiple Kinect cameras; describes an RGB-D computer vision system designed to assist the visually impaired and another for smart-environment sensing to assist elderly and disabled people; examines the effective features that characterize static hand poses and introduces a unified framework to enforce both temporal and spatial constraints for hand parsing; proposes a new classifier architecture for real-time hand pose recognition and a novel hand segmentation and gesture recognition system.

This volume treats the analysis of 3D dynamic scenes using a stereovision system. Several approaches are described, for example two different methods of dealing with long and short sequences of images of an unknown environment, including an arbitrary number of rigid mobile objects. Results obtained from stereovision systems are found to be superior to those from monocular image systems, which are often very sensitive to noise and therefore of little use in practice. It is shown that motion estimation can be further improved by the explicit modelling of uncertainty in geometric objects. The techniques developed in this book have been successfully demonstrated with a large number of real images in the context of visual navigation of a mobile robot.

he problem of analyzing sequences of images to extract three-dimensional T motion and structure has been at the heart of the research in computer vi­ sion for many years. It is very important since its success or failure will determine whether or not vision can be used as a sensory process in reactive systems. The considerable research interest in this field has been motivated at least by the following two points: 1. The redundancy of information contained in time-varying images can over­ come several difficulties encountered in interpreting a single image. 2. There are a lot of important applications including automatic vehicle driv­ ing, traffic control, aerial surveillance, medical inspection and global model construction. However, there are many new problems which should be solved: how to effi­ ciently process the abundant information contained in time-varying images, how to model the change between images, how to model the uncertainty inherently associated with the imaging system and how to solve inverse problems which are generally ill-posed. There are of course many possibilities for attacking these problems and many more remain to be explored. We discuss a few of them in this book based on work carried out during the last five years in the Computer Vision and Robotics Group at INRIA (Institut National de Recherche en Informatique et en Automatique).

he problem of analyzing sequences of images to extract three-dimensional T motion and structure has been at the heart of the research in computer vi­ sion for many years. It is very important since its success or failure will determine whether or not vision can be used as a sensory process in reactive systems. The considerable research interest in this field has been motivated at least by the following two points: 1. The redundancy of information contained in time-varying images can over­ come several difficulties encountered in interpreting a single image. 2. There are a lot of important applications including automatic vehicle driv­ ing, traffic control, aerial surveillance, medical inspection and global model construction. However, there are many new problems which should be solved: how to effi­ ciently process the abundant information contained in time-varying images, how to model the change between images, how to model the uncertainty inherently associated with the imaging system and how to solve inverse problems which are generally ill-posed. There are of course many possibilities for attacking these problems and many more remain to be explored. We discuss a few of them in this book based on work carried out during the last five years in the Computer Vision and Robotics Group at INRIA (Institut National de Recherche en Informatique et en Automatique).

Appendix 164 3. A 3. A. 1 Approximate Estimation of Fundamental Matrix from General Matrix 164 3. A. 2 Estimation of Affine Transformation 165 4 RECOVERY OF EPIPOLAR GEOMETRY FROM LINE SEGMENTS OR LINES 167 Line Segments or Straight Lines 168 4. 1 4. 2 Solving Motion Using Line Segments Between Two Views 173 4. 2. 1 Overlap of Two Corresponding Line Segments 173 Estimating Motion by Maximizing Overlap 175 4. 2. 2 Implementation Details 4. 2. 3 176 Reconstructing 3D Line Segments 4. 2. 4 179 4. 2. 5 Experimental Results 180 4. 2. 6 Discussions 192 4. 3 Determining Epipolar Geometry of Three Views 194 4. 3. 1 Trifocal Constraints for Point Matches 194 4. 3. 2 Trifocal Constraints for Line Correspondences 199 4. 3. 3 Linear Estimation of K, L, and M Using Points and Lines 200 4. 3. 4 Determining Camera Projection Matrices 201 4. 3. 5 Image Transfer 203 4. 4 Summary 204 5 REDEFINING STEREO, MOTION AND OBJECT RECOGNITION VIA EPIPOLAR GEOMETRY 205 5. 1 Conventional Approaches to Stereo, Motion and Object Recognition 205 5. 1. 1 Stereo 205 5. 1. 2 Motion 206 5. 1. 3 Object Recognition 207 5. 2 Correspondence in Stereo, Motion and Object Recognition as 1D Search 209 5. 2. 1 Stereo Matching 209 xi Contents 5. 2. 2 Motion Correspondence and Segmentation 209 5. 2. 3 3D Object Recognition and Localization 210 Disparity and Spatial Disparity Space 210 5.