Multimedia Computing, Communication and Intelligence - Böcker

An uncoded multimedia transmission (UMT) system is one that skips quantization and entropy coding in compression and all subsequent binary operations, including channel coding and bit-to-symbol mapping of modulation. By directly transmitting non-binary symbols with amplitude modulation, the uncoded system avoids the annoying cliff effect observed in the coded transmission system. This advantage makes uncoded transmission more suited to both unicast in varying channel conditions and multicast to heterogeneous users.Particularly, in the first part of Uncoded Multimedia Transmission, we consider how to improve the efficiency of uncoded transmission and make it on par with coded transmission. We then address issues and challenges regarding how to better utilize temporal and spatial correlation of images and video in the uncoded transmission, to achieve the optimal transmission performance. Next, we investigate the resource allocation problem for uncoded transmission, including subchannel, bandwidth and power allocation. By properly allocating these resources, uncoded transmission can achieve higher efficiency and more robust performance. Subsequently, we consider the image and video delivery in MIMO broadcasting networks with diverse channel quality and varying numbers of antennas across receivers. Finally, we investigate the cases where uncoded transmission can be used in conjunction with digital transmission for a balanced efficiency and adaptation capability.This book is the very first monograph in the general area of uncoded multimedia transmission written in a self-contained format. It addresses both the fundamentals and the applications of uncoded transmission. It gives a systematic introduction to the fundamental theory and concepts in this field, and at the same time, also presents specific applications that reveal the great potential and impacts for the technologies generated from the research in this field. By concentrating several important studies and developments currently taking place in the field of uncoded transmission in a single source, this book can reduce the time and cost required to learn and improve skills and knowledge in the field.The authors have been actively working in this field for years, and this book is the final essence of their years of long research in this field. The book may be used as a collection of research notes for researchers in this field, a reference book for practitioners or engineers, as well as a textbook for a graduate advanced seminar in this field or any related fields. The references collected in this book may be used as further reading lists or references for the readers.

An uncoded multimedia transmission (UMT) system is one that skips quantization and entropy coding in compression and all subsequent binary operations, including channel coding and bit-to-symbol mapping of modulation. By directly transmitting non-binary symbols with amplitude modulation, the uncoded system avoids the annoying cliff effect observed in the coded transmission system. This advantage makes uncoded transmission more suited to both unicast in varying channel conditions and multicast to heterogeneous users.Particularly, in the first part of Uncoded Multimedia Transmission, we consider how to improve the efficiency of uncoded transmission and make it on par with coded transmission. We then address issues and challenges regarding how to better utilize temporal and spatial correlation of images and video in the uncoded transmission, to achieve the optimal transmission performance. Next, we investigate the resource allocation problem for uncoded transmission, including subchannel, bandwidth and power allocation. By properly allocating these resources, uncoded transmission can achieve higher efficiency and more robust performance. Subsequently, we consider the image and video delivery in MIMO broadcasting networks with diverse channel quality and varying numbers of antennas across receivers. Finally, we investigate the cases where uncoded transmission can be used in conjunction with digital transmission for a balanced efficiency and adaptation capability.This book is the very first monograph in the general area of uncoded multimedia transmission written in a self-contained format. It addresses both the fundamentals and the applications of uncoded transmission. It gives a systematic introduction to the fundamental theory and concepts in this field, and at the same time, also presents specific applications that reveal the great potential and impacts for the technologies generated from the research in this field. By concentrating several important studies and developments currently taking place in the field of uncoded transmission in a single source, this book can reduce the time and cost required to learn and improve skills and knowledge in the field.The authors have been actively working in this field for years, and this book is the final essence of their years of long research in this field. The book may be used as a collection of research notes for researchers in this field, a reference book for practitioners or engineers, as well as a textbook for a graduate advanced seminar in this field or any related fields. The references collected in this book may be used as further reading lists or references for the readers.

Deep learning has achieved impressive results in image classification, computer vision, and natural language processing. To achieve better performance, deeper and wider networks have been designed, which increase the demand for computational resources. The number of floatingpoint operations (FLOPs) has increased dramatically with larger networks, and this has become an obstacle for convolutional neural networks (CNNs) being developed for mobile and embedded devices. In this context, Binary Neural Networks: Algorithms, Architectures, and Applications will focus on CNN compression and acceleration, which are important for the research community. We will describe numerous methods, including parameter quantization, network pruning, low-rank decomposition, and knowledge distillation. More recently, to reduce the burden of handcrafted architecture design, neural architecture search (NAS) has been used to automatically build neural networks by searching over a vast architecture space. Our book will also introduce NAS and binary NAS and its superiority and state-of-the-art performance in various applications, such as image classification and object detection. We also describe extensive applications of compressed deep models on image classification, speech recognition, object detection, and tracking. These topics can help researchers better understand the usefulness and the potential of network compression on practical applications. Moreover, interested readers should have basic knowledge of machine learning and deep learning to better understand the methods described in this book.Key FeaturesReviews recent advances in CNN compression and accelerationElaborates recent advances on binary neural network (BNN) technologiesIntroduces applications of BNN in image classification, speech recognition, object detection, and more

Deep learning has achieved impressive results in image classification, computer vision, and natural language processing. To achieve better performance, deeper and wider networks have been designed, which increase the demand for computational resources. The number of floatingpoint operations (FLOPs) has increased dramatically with larger networks, and this has become an obstacle for convolutional neural networks (CNNs) being developed for mobile and embedded devices. In this context, Binary Neural Networks: Algorithms, Architectures, and Applications will focus on CNN compression and acceleration, which are important for the research community. We will describe numerous methods, including parameter quantization, network pruning, low-rank decomposition, and knowledge distillation. More recently, to reduce the burden of handcrafted architecture design, neural architecture search (NAS) has been used to automatically build neural networks by searching over a vast architecture space. Our book will also introduce NAS and binary NAS and its superiority and state-of-the-art performance in various applications, such as image classification and object detection. We also describe extensive applications of compressed deep models on image classification, speech recognition, object detection, and tracking. These topics can help researchers better understand the usefulness and the potential of network compression on practical applications. Moreover, interested readers should have basic knowledge of machine learning and deep learning to better understand the methods described in this book.Key FeaturesReviews recent advances in CNN compression and accelerationElaborates recent advances on binary neural network (BNN) technologiesIntroduces applications of BNN in image classification, speech recognition, object detection, and more

AI-Generated Content (AIGC) is a revolutionary engine for digital content generation. In the area of art, AI has achieved remarkable advancements. AI is capable of not only creating paintings or music comparable to human masterpieces, but it also understands and appreciates artwork. For professionals and amateurs, AI is an enabling tool and an opportunity to enjoy a new world of art.This book aims to present the state-of-the-art AI technologies for art creation, understanding, and evaluation. The contents include a survey on cross-modal generation of visual and auditory content, explainable AI and music, AI-enabled robotic theater for Chinese folk art, AI for ancient Chinese music restoration and reproduction, AI for brainwave opera, artistic text style transfer, data-driven automatic choreography, Human-AI collaborative sketching, personalized music recommendation and generation based on emotion and memory (MemoMusic), understanding music and emotion from the brain, music question answering, emotional quality evaluation for generated music, and AI for image aesthetic evaluation.The key features of the book are as follows: AI for Art is a fascinating cross-disciplinary field for the academic community as well as the public. Each chapter is an independent interesting topic, which provides an entry for corresponding readers. It presents SOTA AI technologies for art creation and understanding. The artistry and appreciation of the book is wide-ranging – for example, the combination of AI with traditional Chinese art.This book is dedicated to the international cross-disciplinary AI Art community: professors, students, researchers, and engineers from AI (machine learning, computer vision, multimedia computing, affective computing, robotics, etc.), art (painting, music, dance, fashion, design, etc.), cognitive science, and psychology. General audiences can also benefit from this book.

AI-Generated Content (AIGC) is a revolutionary engine for digital content generation. In the area of art, AI has achieved remarkable advancements. AI is capable of not only creating paintings or music comparable to human masterpieces, but it also understands and appreciates artwork. For professionals and amateurs, AI is an enabling tool and an opportunity to enjoy a new world of art.This book aims to present the state-of-the-art AI technologies for art creation, understanding, and evaluation. The contents include a survey on cross-modal generation of visual and auditory content, explainable AI and music, AI-enabled robotic theater for Chinese folk art, AI for ancient Chinese music restoration and reproduction, AI for brainwave opera, artistic text style transfer, data-driven automatic choreography, Human-AI collaborative sketching, personalized music recommendation and generation based on emotion and memory (MemoMusic), understanding music and emotion from the brain, music question answering, emotional quality evaluation for generated music, and AI for image aesthetic evaluation.The key features of the book are as follows: AI for Art is a fascinating cross-disciplinary field for the academic community as well as the public. Each chapter is an independent interesting topic, which provides an entry for corresponding readers. It presents SOTA AI technologies for art creation and understanding. The artistry and appreciation of the book is wide-ranging – for example, the combination of AI with traditional Chinese art.This book is dedicated to the international cross-disciplinary AI Art community: professors, students, researchers, and engineers from AI (machine learning, computer vision, multimedia computing, affective computing, robotics, etc.), art (painting, music, dance, fashion, design, etc.), cognitive science, and psychology. General audiences can also benefit from this book.

Effective Surveillance for Homeland Security: Balancing Technology and Social Issues provides a comprehensive survey of state-of-the-art methods and tools for the surveillance and protection of citizens and critical infrastructures against natural and deliberate threats. Focusing on current technological challenges involving multi-disciplinary problem analysis and systems engineering approaches, it provides an overview of the most relevant aspects of surveillance systems in the framework of homeland security.Addressing both advanced surveillance technologies and the related socio-ethical issues, the book consists of 21 chapters written by international experts from the various sectors of homeland security. Part I, Surveillance and Society, focuses on the societal dimension of surveillance—stressing the importance of societal acceptability as a precondition to any surveillance system.Part II, Physical and Cyber Surveillance, presents advanced technologies for surveillance. It considers developing technologies that are part of a framework whose aim is to move from a simple collection and storage of information toward proactive systems that are able to fuse several information sources to detect relevant events in their early incipient phase.Part III, Technologies for Homeland Security, considers relevant applications of surveillance systems in the framework of homeland security. It presents real-world case studies of how innovative technologies can be used to effectively improve the security of sensitive areas without violating the rights of the people involved.Examining cutting-edge research topics, the book provides you with a comprehensive understanding of the technological, legislative, organizational, and management issues related to surveillance. With a specific focus on privacy, it presents innovative solutions to many of the issues that remain in the quest to balance security with the preservation of privacy that society demands.

Providing a complete review of existing work in music emotion developed in psychology and engineering, Music Emotion Recognition explains how to account for the subjective nature of emotion perception in the development of automatic music emotion recognition (MER) systems. Among the first publications dedicated to automatic MER, it begins with a comprehensive introduction to the essential aspects of MER—including background, key techniques, and applications.This ground-breaking reference examines emotion from a dimensional perspective. It defines emotions in music as points in a 2D plane in terms of two of the most fundamental emotion dimensions according to psychologists—valence and arousal. The authors present a computational framework that generalizes emotion recognition from the categorical domain to real-valued 2D space. They also: Introduce novel emotion-based music retrieval and organization methods Describe a ranking-base emotion annotation and model training method Present methods that integrate information extracted from lyrics, chord sequence, and genre metadata for improved accuracy Consider an emotion-based music retrieval system that is particularly useful for mobile devices The book details techniques for addressing the issues related to: the ambiguity and granularity of emotion description, heavy cognitive load of emotion annotation, subjectivity of emotion perception, and the semantic gap between low-level audio signal and high-level emotion perception. Complete with more than 360 useful references, 12 example MATLAB® codes, and a listing of key abbreviations and acronyms, this cutting-edge guide supplies the technical understanding and tools needed to develop your own automatic MER system based on the automatic recognition model.

The rapid advancement of digital multimedia technologies has not only revolutionized the production and distribution of audiovisual content, but also created the need to efficiently analyze TV programs to enable applications for content managers and consumers. Leaving no stone unturned, TV Content Analysis: Techniques and Applications provides a detailed exploration of TV program analysis techniques.Leading researchers and academics from around the world supply scientifically sound treatment of recent developments across the related subject areas—including systems, architectures, algorithms, applications, research results, emerging approaches, and open issues. The book is organized into six parts: Content Extraction - deals with automatic analysis and annotation of TV content, addressing generic semantics and concepts as well as TV contentContent Structuring - examines techniques for identifying interesting parts of TV programs and supplying direct access to itContent Recommendation - explores the problem of providing users with the most relevant content, addressing the problem of an ever-increasing amount of available content Content Quality - considers visual perception and quality approaches in the multi-display TV context and the specific mobile TV scenarioWeb and Social TV - presents studies on Web and TV convergence and on how user-generated content in Web 2.0 applications can be used to enhance servicesContent Production - covers postproduction, visual effects, and presentation standardsMost parts start with a chapter that provides an overview of that area, followed by state-of-the-art approaches focusing on specific issues covered in that section. Reporting on recent advances in the field, the book provides you with the global view and up-to-date understanding of emerging trends needed to participate in the development of the digital TV domain.

Effective Surveillance for Homeland Security: Balancing Technology and Social Issues provides a comprehensive survey of state-of-the-art methods and tools for the surveillance and protection of citizens and critical infrastructures against natural and deliberate threats. Focusing on current technological challenges involving multi-disciplinary problem analysis and systems engineering approaches, it provides an overview of the most relevant aspects of surveillance systems in the framework of homeland security.Addressing both advanced surveillance technologies and the related socio-ethical issues, the book consists of 21 chapters written by international experts from the various sectors of homeland security. Part I, Surveillance and Society, focuses on the societal dimension of surveillance—stressing the importance of societal acceptability as a precondition to any surveillance system.Part II, Physical and Cyber Surveillance, presents advanced technologies for surveillance. It considers developing technologies that are part of a framework whose aim is to move from a simple collection and storage of information toward proactive systems that are able to fuse several information sources to detect relevant events in their early incipient phase.Part III, Technologies for Homeland Security, considers relevant applications of surveillance systems in the framework of homeland security. It presents real-world case studies of how innovative technologies can be used to effectively improve the security of sensitive areas without violating the rights of the people involved.Examining cutting-edge research topics, the book provides you with a comprehensive understanding of the technological, legislative, organizational, and management issues related to surveillance. With a specific focus on privacy, it presents innovative solutions to many of the issues that remain in the quest to balance security with the preservation of privacy that society demands.