Liqiang Nie – författare

Nowadays, fashion has become an essential aspect of people''s daily life. As each outfit usually comprises several complementary items, such as a top, bottom, shoes, and accessories, a proper outfit largely relies on the harmonious matching of these items. Nevertheless, not everyone is good at outfit composition, especially those who have a poor fashion aesthetic. Fortunately, in recent years the number of online fashion-oriented communities, like IQON and Chictopia, as well as e-commerce sites, like Amazon and eBay, has grown. The tremendous amount of real-world data regarding people''s various fashion behaviors has opened a door to automatic clothing matching.

Despite its significant value, compatibility modeling for clothing matching that assesses the compatibility score for a given set of (equal or more than two) fashion items, e.g., a blouse and a skirt, yields tough challenges: (a) the absence of comprehensive benchmark; (b) comprehensive compatibility modeling with the multi-modal feature variables is largely untapped; (c) how to utilize the domain knowledge to guide the machine learning; (d) how to enhance the interpretability of the compatibility modeling; and (e) how to model the user factor in the personalized compatibility modeling. These challenges have been largely unexplored to date.

In this book, we shed light on several state-of-the-art theories on compatibility modeling. In particular, to facilitate the research, we first build three large-scale benchmark datasets from different online fashion websites, including IQON and Amazon. We then introduce a general data-driven compatibility modeling scheme based on advanced neural networks. To make use of the abundant fashion domain knowledge, i.e., clothing matching rules, we next present a novel knowledge-guided compatibility modeling framework. Thereafter, to enhance the model interpretability, we put forward a prototype-wise interpretable compatibility modeling approach. Following that,noticing the subjective aesthetics of users, we extend the general compatibility modeling to the personalized version. Moreover, we further study the real-world problem of personalized capsule wardrobe creation, aiming to generate a minimum collection of garments that is both compatible and suitable for the user. Finally, we conclude the book and present future research directions, such as the generative compatibility modeling, virtual try-on with arbitrary poses, and clothing generation.

This Third Edition sheds light on state-of-the-art theories and practices in multimodal compatibility modeling and recommendation, offering comprehensive insights into this evolving field. This topic, and fashion compatibility modeling in particular, has garnered increasing research attention in recent years due to the significant economic impact of e-commerce. Building upon recent research and the prior edition, the authors present a series of graph-learning based multimodal compatibility modeling schemes, all of which have been proven to be effective over several public real-world datasets. This book introduces a number of advanced multimodal compatibility modeling and recommendation methods, including category-guided multimodal compatibility modeling and try-on-guided multimodal compatibility modeling. The authors also provide comprehensive solutions, including correlation-oriented graph learning, modality-oriented graph learning, unsupervised disentangled graph learning, partially supervised disentangled graph learning, and metapath-guided heterogeneous graph learning. 

This book presents an in-depth exploration of multimodal learning toward recommendation, along with a comprehensive survey of the most important research topics and state-of-the-art methods in this area.

First, it presents a semantic-guided feature distillation method which employs a teacher-student framework to robustly extract effective recommendation-oriented features from generic multimodal features. Next, it introduces a novel multimodal attentive metric learning method to model user diverse preferences for various items. Then it proposes a disentangled multimodal representation learning recommendation model, which can capture users’ fine-grained attention to different modalities on each factor in user preference modeling. Furthermore, a meta-learning-based multimodal fusion framework is developed to model the various relationships among multimodal information. Building on the success of disentangled representation learning, it further proposes an attribute-driven disentangled representation learning method, which uses attributes to guide the disentanglement process in order to improve the interpretability and controllability of conventional recommendation methods. Finally, the book concludes with future research directions in multimodal learning toward recommendation.

The book is suitable for graduate students and researchers who are interested in multimodal learning and recommender systems. The multimodal learning methods presented are also applicable to other retrieval or sorting related research areas, like image retrieval, moment localization, and visual question answering.

This book systematically examines scalability and effectiveness challenges related to the application of graph convolutional networks (GCNs) in recommender systems. By effectively modeling graph structures, GCNs excel in capturing high-order relationships between users and items, enabling the creation of enriched and expressive representations.

The book focuses on two overarching problem categories: the first area deals with problems specific to GCN-based recommendation models, including over-smoothing, noisy neighboring nodes, and interpretability limitations. The second one encompasses broader challenges in recommendation systems that GCN-based methods are particularly well-suited to address as the attribute missing problem or feature misalignment. Through rigorous exploration of these challenges, this book presents innovative GCN-based solutions to push the boundaries of recommender system design. To this end, techniques such as interest-aware message-passing strategy, cluster-based collaborative filtering, semantic aspects extraction, attribute-aware attention mechanisms, and light graph transformer are presented.

Each chapter combines theoretical insights with practical implementations and experimental validation, offering a comprehensive resource for researchers, advanced professionals, and graduate students alike.

This book provides a comprehensive exploration of video moment localization, a rapidly emerging research field focused on enabling precise retrieval of specific moments within untrimmed, unsegmented videos. With the rapid growth of digital content and the rise of video-sharing platforms, users face significant challenges when searching for particular content across vast video archives. This book addresses how video moment localization uses natural language queries to bridge the gap between video content and semantic understanding, offering an intuitive solution for locating specific moments across diverse domains like surveillance, education, and entertainment.

 This book explores the latest advancements in video moment localization, addressing key issues such as accuracy, efficiency, and scalability. It presents innovative techniques for contextual understanding and cross-modal semantic alignment, including attention mechanisms and dynamic query decomposition. Additionally, the book discusses solutions for enhancing computational efficiency and scalability, such as semantic pruning and efficient hashing, while introducing frameworks for better integration between visual and textual data. It also examines weakly-supervised learning approaches to reduce annotation costs without sacrificing performance. Finally, the book covers real-world applications and offers insights into future research directions.