Bogdan Ionescu - Böcker

Recent years have witnessed important advancements in our understanding of the psychological underpinnings of subjective properties of visual information, such as aesthetics, memorability, or induced emotions. Concurrently, computational models of objective visual properties such as semantic labelling and geometric relationships have made significant breakthroughs using the latest achievements in machine learning and large-scale data collection. There has also been limited but important work exploiting these breakthroughs to improve computational modelling of subjective visual properties. The time is ripe to explore how advances in both of these fields of study can be mutually enriching and lead to further progress.This book combines perspectives from psychology and machine learning to showcase a new,  unified  understanding  of  how  images  and  videos  influence  high-level  visual  perception  -  particularly  interestingness,  affective  values  and  emotions,  aesthetic  values,  memorability,  novelty,  complexity,  visual composition and stylistic attributes, and creativity. These human-based metrics are interesting for a very broad range of current applications, ranging from content retrieval and search, storytelling, to targeted advertising, education and learning, and content filtering.Work already exists in the literature that studies the psychological aspects of these notions or investigates potential correlations between two or more of these human concepts. Attempts at building computational models capable of predicting such notions can also be found, using state-of-the-art machine learning techniques. Nevertheless their performance proves that there is still room for improvement, as the tasks are by nature highly challenging and multifaceted, requiring thought on both the psychological implications of the human concepts, as well as their translation to machines.

Recent years have witnessed important advancements in our understanding of the psychological underpinnings of subjective properties of visual information, such as aesthetics, memorability, or induced emotions. Concurrently, computational models of objective visual properties such as semantic labelling and geometric relationships have made significant breakthroughs using the latest achievements in machine learning and large-scale data collection. There has also been limited but important work exploiting these breakthroughs to improve computational modelling of subjective visual properties. The time is ripe to explore how advances in both of these fields of study can be mutually enriching and lead to further progress.This book combines perspectives from psychology and machine learning to showcase a new,  unified  understanding  of  how  images  and  videos  influence  high-level  visual  perception  -  particularly  interestingness,  affective  values  and  emotions,  aesthetic  values,  memorability,  novelty,  complexity,  visual composition and stylistic attributes, and creativity. These human-based metrics are interesting for a very broad range of current applications, ranging from content retrieval and search, storytelling, to targeted advertising, education and learning, and content filtering.Work already exists in the literature that studies the psychological aspects of these notions or investigates potential correlations between two or more of these human concepts. Attempts at building computational models capable of predicting such notions can also be found, using state-of-the-art machine learning techniques. Nevertheless their performance proves that there is still room for improvement, as the tasks are by nature highly challenging and multifaceted, requiring thought on both the psychological implications of the human concepts, as well as their translation to machines.

This book constitutes the refereed proceedings of the 12th International Conference of the CLEF Association, CLEF 2021, held virtually in September 2021.The conference has a clear focus on experimental information retrieval with special attention to the challenges of multimodality, multilinguality, and interactive search ranging from unstructured to semi structures and structured data.The 11 full papers presented in this volume were carefully reviewed and selected from 21 submissions. This year, the contributions addressed the following challenges: application of neural methods for entity recognition as well as misinformation detection in the health area, skills extraction in job-match databases, stock market prediction using financial news, and extraction of audio features for podcast retrieval.In addition to this, the volume presents 5 “best of the labs” papers which were reviewed as full paper submissions with the same review criteria. 12 lab overview papers were accepted and represent scientific challenges based on new data sets and real world problems in multimodal and multilingual information access.

This book presents a thorough overview of fusion in computer vision, from an interdisciplinary and multi-application viewpoint, describing successful approaches, evaluated in the context of international benchmarks that model realistic use cases. Features: examines late fusion approaches for concept recognition in images and videos; describes the interpretation of visual content by incorporating models of the human visual system with content understanding methods; investigates the fusion of multi-modal features of different semantic levels, as well as results of semantic concept detections, for example-based event recognition in video; proposes rotation-based ensemble classifiers for high-dimensional data, which encourage both individual accuracy and diversity within the ensemble; reviews application-focused strategies of fusion in video surveillance, biomedical information retrieval, and content detection in movies; discusses the modeling of mechanisms of human interpretation of complex visual content.