International Series in Video Computing – serie

The recent development of intelligent surveillance systems has captured the interest of both academic research labs and industry. Automated Multi-Camera Surveillance addresses monitoring of people and vehicles, and detection of threatening objects and events in a variety of scenarios. In this book, techniques for development of an automated multi-camera surveillance system are discussed and proposed. The state-of-the-art in the automated surveillance systems is reviewed as well. Detailed explanation of sub-components of surveillance systems are provided, and enhancements to each of these components are proposed. The authors identify important challenges that such a system must address, and propose solutions. Development of a specific surveillance system called “KNIGHT” is described, along with the authors’ experience using it. This book enables the reader to understand the mathematical models and algorithms underlying automated surveillance as well as the benefits and limitations of using such methods.

With the ubiquity of new information technology and media, more effective and friendly methods for human computer interaction (HCI) are being developed which do not rely on traditional devices such as keyboards, mice and displays. The first step for any intelligent HCI system is face detection, and one of most friendly HCI systems is hand gesture. Face Detection and Gesture Recognition for Human-Computer Interaction introduces the frontiers of vision-based interfaces for intelligent human computer interaction with focus on two main issues: face detection and gesture recognition. The first part of the book reviews and discusses existing face detection methods, followed by a discussion on future research. Performance evaluation issues on the face detection methods are also addressed. The second part discusses an interesting hand gesture recognition method based on a generic motion segmentation algorithm. The system has been tested with gestures from American Sign Language with promising results. We conclude this book with comments on future work in face detection and hand gesture recognition.Face Detection and Gesture Recognition for Human-Computer Interaction will interest those working in vision-based interfaces for intelligent human computer interaction. It also contains a comprehensive survey on existing face detection methods, which will serve as the entry point for new researchers embarking on such topics. Furthermore, this book also covers in-depth discussion on motion segmentation algorithms and applications, which will benefit more seasoned graduate students or researchers interested in motion pattern recognition.

This text focuses on visual event detection. It demonstrates that computer vision research has matured to a point where meaningful visual event detection can be achieved. The authors propose that the exact object and motion information is not necessary to achieve video event detection. They show that some visual events are sufficiently described by little more than the broad categories of the constituent objects and their qualitative motions. The Video Computing book series provides a forum for the dissemination of innovative research results for computer vision, image processing, database and computer graphics researchers.

This text describes some computer vision-based methods that analyze video sequences of humans. More specifically, methods for tracking multiple humans in a scene, estimating postures of a human body in 3D in real-time, and recognizing a person's behaviour (gestures or activities) are discussed. For the tracking algorithm, the authors developed a non-synchronous method that tracks multiple persons by exploiting a Kalman filter that is applied to multiple video sequences. For estimating postures, an algorithm is presented that locates the significant points which determine postures of a human body, in 3D in real-time. Human activities are recognized from a video sequence by the HMM (Hidden Markov Models)-based method that the authors pioneered. The effectiveness of the three methods is shown by experimental results. The posture estimation method described in this book is a world-leading method in that it can estimate postures in 3D in real-time. As described in this book, the posture estimation method can be applied to avatar-based telecommunication systems, which require realistic, real-time reproduction of human images.This work should prove a useful reference for both professional and academic researchers in the fields of computer vision and image processing, also, to those working in video surveillance and monitoring, virtual reality, computer graphics, pattern recognition, telecommunication, human-computer interface, and general computer science. It should also be suitable for use in graduate classes in computer vision or image processing.

With the explosion of digital media and online services, a key challenge is the automation of content annotation, indexing and organizing for efficient media access, retrieval and browsing. A major failing of automated media annotation systems is the semantic gap - the incompatibility between the low-level features that can be computed to describe media and the high-level meaning sought by user queries in searches for audio and video. This text explores the area of computational media aesthetics to bridge the semantic gap. It brings together researchers from multiple disciplines, from computer scientists to content creators from engineers to media specialists, to highlight advances in this area. It examines the elements of media production and aesthetics principles in order to establish the foundation for semantics indexing, and to build innovative and effective technologies for content annotation, search and browsing. The book outlines strategic means and methods to derive high level semantic constructs from automatic analysis of media. It is suitable for the use as a text in courses on multimedia and informational retrieval.

Multimodal Video Characterization and Summarization is a valuable research tool for both professionals and academicians working in the video field.This book describes the methodology for using multimodal audio, image, and text technology to characterize video content. This new and groundbreaking science has led to many advances in video understanding, such as the development of a video summary. Applications and methodology for creating video summaries are described, as well as user-studies for evaluation and testing.

Written by international experts in the field, "Video Registration" is a survey of the state of the art in this fundamental technology. It is a reference for both academicians and practicing engineers working in the fields of security, surveillance, multimedia and robotics. The increased availability of low-cost, low-power, highly accurate video imagery has resulted in a rapid growth of the applications for this data. Video imagery has many advantages over still frame imagery; for example, it provides context and timing relationships, which are suitable for dynamic situation monitoring and action verification. Manipulation of video requires automatic processing and analysis (computer vision and image processing), vast amounts of storage and efficient search methods (databases), high bandwidth communication (networking), and real-time implementations (VLSI/hardware).Users of video imagery include disaster relief agencies, environmental monitoring and planning applications, tactical military groups, civilian agencies such as homeland security agencies, city planners, transportation (traffic management), the entertainment industry, law enforcement groups, landscape ecologists, WWW users and trainers and educators. "Video Registration" is suitable as a text at the senior undergraduate and graduate levels.

This text aims to be a reference for the practitioners and academicians in the fields of multimedia search engines. Half a terabyte or 9000 hours of motion pictures are produced around the world every year. Furthermore, 3000 television stations broadcasting for twenty-four hours a day produce eight million hours per year, amounting to 24,000 terabytes of data. Although some of the data is labelled at the time of production, an enormous portion remains unindexed. For practical access to such huge amounts of data, there is a great need to develop efficient tools for browsing and retrieving content of interest, so that producers and end users can quickly locate specific video sequences in this ocean of audio-visual data. This text describes the main techniques being developed by the major players in industry and academic research to address this problem.

"Exploration of Visual Data" presents research efforts in the area of content-based exploration of image and video data. The main objective is to bridge the semantic gap between high-level concepts in the human mind and low-level features extractable by the machines. The two key issues emphasized are "content-awareness" and "user-in-the-loop". The authors provide a comprehensive review on algorithms for visual feature extraction based on colour, texture, shape and structure, and techniques for incorporating such information to aid browsing, exploration, search and streaming of image and video data. They also discuss issues related to the mixed use of textual and low-level visual features to facilitate more effective access of multimedia data. To bridge the semantic gap, significant recent research efforts have also been put on learning during user interactions, which is also known as "relevance feedback". The difficulty and challenge also come from the personalized information need of each user and a small amount of feedbacks the machine could obtain through real-time user interaction.The authors present and discuss several recently proposed classification and learning techniques that are specifically designed for this problem, with kernel- and boosting-based approaches for nonlinear extensions. "Exploration of Visual Data" provides state-of-the-art materials on the topics of content-based description of visual data, content-based low-bitrate video streaming, and latest asymmetric and nonlinear relevance feedback algorithms, which to date are unpublished.

This monograph is an essential reference for both academic and professional researchers working in the fields of computer vision, surveillance, HCI, image processing, and modeling. The human face provides important visual cues for human-human communication and human-computer interaction (HCI). With the advances in new information techniques and media, 3D face processing techniques will help an intelligent HCI system interpret and deliver more natural multimedia information related to human faces. 3D Face Processing: Modeling, Analysis and Synthesis introduces the frontiers of 3D face processing techniques. It reviews existing 3D face processing techniques, including techniques for 3D face geometry modeling; 3D face motion modeling; and 3D face motion tracking and animation. Then it discusses a unified framework for face modeling, analysis and synthesis. In this framework, the authors present new methods for modeling complex natural facial motion, as well as face appearance variations due to illumination and subtle motion. Then the authors apply the framework to face tracking, expression recognition and face avatar for HCI interface.They conclude this book with comments on future work in the 3D face processing framework. 3D Face Processing: Modeling, Analysis and Synthesis will interest those working in face processing for intelligent human computer interaction and video surveillance. It contains a comprehensive survey on existing face processing techniques, which can serve as a reference for students and researchers. It also covers in-depth discussion on face motion analysis and synthesis algorithms, which will benefit more advanced graduate students and researchers.

The recent development of intelligent surveillance systems has captured the interest of both academic research labs and industry. Automated Multi-Camera Surveillance addresses monitoring of people and vehicles, and detection of threatening objects and events in a variety of scenarios. In this book, techniques for development of an automated multi-camera surveillance system are discussed and proposed. The state-of-the-art in the automated surveillance systems is reviewed as well. Detailed explanation of sub-components of surveillance systems are provided, and enhancements to each of these components are proposed. The authors identify important challenges that such a system must address, and propose solutions. Development of a specific surveillance system called “KNIGHT” is described, along with the authors’ experience using it. This book enables the reader to understand the mathematical models and algorithms underlying automated surveillance as well as the benefits and limitations of using such methods.

This book is one of the first books to focus on visual event detection. It demonstrates that computer vision research has matured to a point where meaningful visual event detection can be achieved. The authors propose that the exact object and motion information is not necessary to achieve video event detection. They show that some visual events are sufficiently described by little more than the broad categories of the constituent objects and their qualitative motions. The Video Computing book series provides a forum for the dissemination of innovative research results for computer vision, image processing, database and computer graphics researchers. Visual Event Detection will be of interest to those working in video analysis, video understanding, video compression, image understanding, and artificial intelligence.

Multimodal Video Characterization and Summarization is a valuable research tool for both professionals and academicians working in the video field.This book describes the methodology for using multimodal audio, image, and text technology to characterize video content. This new and groundbreaking science has led to many advances in video understanding, such as the development of a video summary. Applications and methodology for creating video summaries are described, as well as user-studies for evaluation and testing.

Video Mining is an essential reference for the practitioners and academicians in the fields of multimedia search engines. Half a terabyte or 9,000 hours of motion pictures are produced around the world every year. Furthermore, 3,000 television stations broadcasting for twenty-four hours a day produce eight million hours per year, amounting to 24,000 terabytes of data. Although some of the data is labeled at the time of production, an enormous portion remains unindexed. For practical access to such huge amounts of data, there is a great need to develop efficient tools for browsing and retrieving content of interest, so that producers and end users can quickly locate specific video sequences in this ocean of audio-visual data. Video Mining is important because it describes the main techniques being developed by the major players in industry and academic research to address this problem. It is the first time research from these leaders in the field developing the next-generation multimedia search engines is being described in great detail and gathered into a single volume.Video Mining will give valuable insights to all researchers and non-specialists who want to understand the principles applied by the multimedia search engines that are about to be deployed on the Internet, in studios' multimedia asset management systems, and in video-on-demand systems.

3D Face Processing: Modeling, Analysis and Synthesis introduces the frontiers of 3D face processing techniques. It reviews existing 3D face processing techniques, including techniques for 3D face geometry modeling; 3D face motion modeling; and 3D face motion tracking and animation. Then it discusses a unified framework for face modeling, analysis and synthesis. In this framework, the authors present new methods for modeling complex natural facial motion, as well as face appearance variations due to illumination and subtle motion. Then the authors apply the framework to face tracking, expression recognition and face avatar for HCI interface. They conclude this book with comments on future work in the 3D face processing framework. 3D Face Processing: Modeling, Analysis and Synthesis will interest those working in face processing for intelligent human computer interaction and video surveillance. It contains a comprehensive survey on existing face processing techniques, which can serve as a reference for students and researchers. It also covers in-depth discussion on face motion analysis and synthesis algorithms, which will benefit more advanced graduate students and researchers.

Traditionally, scientific fields have defined boundaries, and scientists work on research problems within those boundaries. However, from time to time those boundaries get shifted or blurred to evolve new fields. For instance, the original goal of computer vision was to understand a single image of a scene, by identifying objects, their structure, and spatial arrangements. This has been referred to as image understanding. Recently, computer vision has gradually been making the transition away from understanding single images to analyz­ ing image sequences, or video understanding. Video understanding deals with understanding of video sequences, e. g. , recognition of gestures, activities, fa­ cial expressions, etc. The main shift in the classic paradigm has been from the recognition of static objects in the scene to motion-based recognition of actions and events. Video understanding has overlapping research problems with other fields, therefore blurring the fixed boundaries. Computer graphics, image processing, and video databases have obvious overlap with computer vision. The main goal of computer graphics is to gener­ ate and animate realistic looking images, and videos. Researchers in computer graphics are increasingly employing techniques from computer vision to gener­ ate the synthetic imagery. A good example of this is image-based rendering and modeling techniques, in which geometry, appearance, and lighting is de­ rived from real images using computer vision techniques. Here the shift is from synthesis to analysis followed by synthesis.

Exploration of Visual Data presents latest research efforts in the area of content-based exploration of image and video data. The main objective is to bridge the semantic gap between high-level concepts in the human mind and low-level features extractable by the machines.The two key issues emphasized are "content-awareness" and "user-in-the-loop". The authors provide a comprehensive review on algorithms for visual feature extraction based on color, texture, shape, and structure, and techniques for incorporating such information to aid browsing, exploration, search, and streaming of image and video data. They also discuss issues related to the mixed use of textual and low-level visual features to facilitate more effective access of multimedia data.Exploration of Visual Data provides state-of-the-art materials on the topics of content-based description of visual data, content-based low-bitrate video streaming, and latest asymmetric and nonlinear relevance feedback algorithms, which to date are unpublished.

Analyzing Video Sequences of Multiple Humans: Tracking, Posture Estimation and Behavior Recognition describes some computer vision-based methods that analyze video sequences of humans. More specifically, methods for tracking multiple humans in a scene, estimating postures of a human body in 3D in real-time, and recognizing a person's behavior (gestures or activities) are discussed. For the tracking algorithm, the authors developed a non-synchronous method that tracks multiple persons by exploiting a Kalman filter that is applied to multiple video sequences. For estimating postures, an algorithm is presented that locates the significant points which determine postures of a human body, in 3D in real-time. Human activities are recognized from a video sequence by the HMM (Hidden Markov Models)-based method that the authors pioneered. The effectiveness of the three methods is shown by experimental results.

Traditionally, scientific fields have defined boundaries, and scientists work on research problems within those boundaries. However, from time to time those boundaries get shifted or blurred to evolve new fields. For instance, the original goal of computer vision was to understand a single image of a scene, by identifying objects, their structure, and spatial arrangements. This has been referred to as image understanding. Recently, computer vision has gradually been making the transition away from understanding single images to analyz­ ing image sequences, or video understanding. Video understanding deals with understanding of video sequences, e. g. , recognition of gestures, activities, fa­ cial expressions, etc. The main shift in the classic paradigm has been from the recognition of static objects in the scene to motion-based recognition of actions and events. Video understanding has overlapping research problems with other fields, therefore blurring the fixed boundaries. Computer graphics, image processing, and video databases have obvious overlap with computer vision. The main goal of computer graphics is to gener­ ate and animate realistic looking images, and videos. Researchers in computer graphics are increasingly employing techniques from computer vision to gen­ erate the synthetic imagery. A good example of this is image-based rendering and modeling techniques, in which geometry, appearance, and lighting is de­ rived from real images using computer vision techniques. Here the shift is from synthesis to analysis followed by synthesis.

Traditionally, scientific fields have defined boundaries, and scientists work on research problems within those boundaries. However, from time to time those boundaries get shifted or blurred to evolve new fields. For instance, the original goal of computer vision was to understand a single image of a scene, by identifying objects, their structure, and spatial arrangements. This has been referred to as image understanding. Recently, computer vision has gradually been making the transition away from understanding single images to analyzing image sequences, or video understanding. Video understanding deals with understanding of video sequences, e. g. , recognition of gestures, activities, facial expressions, etc. The main shift in the classic paradigm has been from the recognition of static objects in the scene to motion-based recognition of actions and events. Video understanding has overlapping research problems with other fields, therefore blurring the fixed boundaries. Computer graphics, image processing, and video databases have obvious overlap with computer vision. The main goal of computer graphics is to gener­ ate and animate realistic looking images, and videos. Researchers in computer graphics are increasingly employing techniques from computer vision to gen­ erate the synthetic imagery. A good example of this is image-based rendering and modeling techniques, in which geometry, appearance, and lighting is de­ rived from real images using computer vision techniques. Here the shift is from synthesis to analysis followed by synthesis.

Over the last several years there has been a growing interest in developing computational methodologies for modeling and analyzing movements and behaviors of ‘crowds' of people. This interest spans several scientific areas that includes Computer Vision, Computer Graphics, and Pedestrian Evacuation Dynamics. Despite the fact that these different scientific fields are trying to model the same physical entity (i.e. a crowd of people), research ideas have evolved independently. As a result each discipline has developed techniques and perspectives that are characteristically their own. The goal of this book is to provide the readers a comprehensive map towards the common goal of better analyzing and synthesizing the pedestrian movement in dense, heterogeneous crowds. The book is organized into different parts that consolidate various aspects of research towards this common goal, namely the modeling, simulation, and visual analysis of crowds. Through this book, readers will see the common ideas and vision as well as the different challenges and techniques, that will stimulate novel approaches to fully grasping “crowds."

Over the last several years there has been a growing interest in developing computational methodologies for modeling and analyzing movements and behaviors of ‘crowds' of people. This interest spans several scientific areas that includes Computer Vision, Computer Graphics, and Pedestrian Evacuation Dynamics. Despite the fact that these different scientific fields are trying to model the same physical entity (i.e. a crowd of people), research ideas have evolved independently. As a result each discipline has developed techniques and perspectives that are characteristically their own. The goal of this book is to provide the readers a comprehensive map towards the common goal of better analyzing and synthesizing the pedestrian movement in dense, heterogeneous crowds. The book is organized into different parts that consolidate various aspects of research towards this common goal, namely the modeling, simulation, and visual analysis of crowds. Through this book, readers will see the common ideas and vision as well as the different challenges and techniques, that will stimulate novel approaches to fully grasping “crowds."

Various fundamental applications in computer vision and machine learning require finding the basis of a certain subspace. Examples of such applications include face detection, motion estimation, and activity recognition. An increasing interest has been recently placed on this area as a result of significant advances in the mathematics of matrix rank optimization. Interestingly, robust subspace estimation can be posed as a low-rank optimization problem, which can be solved efficiently using techniques such as the method of Augmented Lagrange Multiplier. In this book, the authors discuss fundamental formulations and extensions for low-rank optimization-based subspace estimation and representation. By minimizing the rank of the matrix containing observations drawn from images, the authors demonstrate  how to solve four fundamental computer vision problems, including video denosing, background subtraction, motion estimation, and activity recognition.