Wolfgang Minker - Böcker

The development of computer and telecommunication technologies led to a revolutioninthewaythatpeopleworkandcommunicatewitheachother.One of the results is that large amount of information will increasingly be held in a form that is natural for users, as speech in natural language. In the presented work, we investigate the speech signal capture problem, which includes the separation of multiple interfering speakers using microphone arrays. Adaptive beamforming is a classical approach which has been developed since the seventies. However it requires a double-talk detector (DTD) that interrupts the adaptation when the target is active, since otherwise target cancelation occurs. The fact that several speakers may be active simulta- ouslymakesthisdetectiondi?cult,andifadditionalbackgroundnoiseoccurs, even less reliable. Our proposed approaches address this separation problem using continuous, uninterrupted adaptive algorithms. The advantage seems twofold:Firstly,thealgorithmdevelopmentismuchsimplersincenodetection mechanism needs to be designed and no threshold is to be tuned. Secondly, the performance may be improved due to the adaptation during periods of double-talk. In the ?rst part of the book, we investigate a modi?cation of the widely usedNLMSalgorithm,termedImplicitLMS(ILMS),whichimplicitlyincludes an adaptation control and does not require any threshold. Experimental ev- uations reveal that ILMS mitigates the target signal cancelation substantially with the distributed microphone array. However, in the more di?cult case of the compact microphone array, this algorithm does not su?ciently reduce the target signal cancelation. In this case, more sophisticated blind source se- ration techniques (BSS) seem necessary.

Bandwidth Extension of Speech Signals describes the theory and methods for quality enhancement of clean speech signals and distorted speech signals such as those that have undergone a band limitation, for instance, in a telephone network. Problems and the respective solutions are discussed for the different approaches. The different approaches are evaluated and a real-time implementation of the most promising approach is presented. The book includes topics related to speech coding, pattern- / speech recognition, speech enhancement, statistics and digital signal processing in general.

Over the last decade a number of research areas have contributed to the concept of advanced intelligent environments, these include ubiquitous computing, pervasive computing, embedded intelligence, intelligent user interfaces, human factors, intelligent buildings, mobile communications, domestic robots, intelligent sensors, artistic and architectural design and ambient intelligence. Undeniably, multimodal spoken language dialogue interaction is a key factor in ensuring natural interaction and therefore of particular interest for advanced intelligent environments. The book covers all the key topics in the field of intelligent environments from a variety of leading researchers.

This text investigates the problem of automatic natural language understanding in a spoken language dialogue system. The focus is on the design of a stochastic parser and its evaluation with respect to a conventional rule-based method. It should be of interest to researchers in artificial intelligence, especially those in natural language processing, computational linguistics, and speech recognition. It should also appeal to practising engineers who work in the area of interactive speech systems.

This book is based on publications from the ISCA Tutorial and Research Workshop on Multi-Modal Dialogue in Mobile Environments held at Kloster Irsee, Germany, in 2002. The workshop covered various aspects of devel- ment and evaluation of spoken multimodal dialogue systems and components with particular emphasis on mobile environments, and discussed the state-- the-art within this area. On the development side the major aspects addressed include speech recognition, dialogue management, multimodal output gene- tion, system architectures, full applications, and user interface issues. On the evaluation side primarily usability evaluation was addressed. A number of high quality papers from the workshop were selected to form the basis of this book. The volume is divided into three major parts which group together the ov- all aspects covered by the workshop. The selected papers have all been - tended, reviewed and improved after the workshop to form the backbone of the book. In addition, we have supplemented each of the three parts by an invited contribution intended to serve as an overview chapter.

This book is based on publications from the ISCA Tutorial and Research Workshop on Multi-Modal Dialogue in Mobile Environments held at Kloster Irsee, Germany, in 2002. The workshop covered various aspects of devel- ment and evaluation of spoken multimodal dialogue systems and components with particular emphasis on mobile environments, and discussed the state-- the-art within this area. On the development side the major aspects addressed include speech recognition, dialogue management, multimodal output gene- tion, system architectures, full applications, and user interface issues. On the evaluation side primarily usability evaluation was addressed. A number of high quality papers from the workshop were selected to form the basis of this book. The volume is divided into three major parts which group together the ov- all aspects covered by the workshop. The selected papers have all been - tended, reviewed and improved after the workshop to form the backbone of the book. In addition, we have supplemented each of the three parts by an invited contribution intended to serve as an overview chapter.

In its nine chapters, this book provides an overview of the state-of-the-art and best practice in several sub-fields of evaluation of text and speech systems and components. The evaluation aspects covered include speech and speaker recognition, speech synthesis, animated talking agents, part-of-speech tagging, parsing, and natural language software like machine translation, information retrieval, question answering, spoken dialogue systems, data resources, and annotation schemes. With its broad coverage and original contributions this book is unique in the field of evaluation of speech and language technology.This book is of particular relevance to advanced undergraduate students, PhD students, academic and industrial researchers, and practitioners.

In its nine chapters, this book provides an overview of the state-of-the-art and best practice in several sub-fields of evaluation of text and speech systems and components. The evaluation aspects covered include speech and speaker recognition, speech synthesis, animated talking agents, part-of-speech tagging, parsing, and natural language software like machine translation, information retrieval, question answering, spoken dialogue systems, data resources, and annotation schemes. With its broad coverage and original contributions this book is unique in the field of evaluation of speech and language technology.This book is of particular relevance to advanced undergraduate students, PhD students, academic and industrial researchers, and practitioners.

This book edition highlights recent trends and important issues that still remain only partially solved or even unsolved within the broad ?eld of discourse and dialogue. The ?eld is discussed and illustrated both from an overall spoken (multimodal) dialogue system perspective as well as from a more compone- related perspective. Issues discussed include, for example, discourse and dialogue modelling in research versus industrial spoken dialogue systems, evaluation, miscommunication and error handling, grounding, statistical and corpus-based approaches to discourse and dialogue modelling, data analysis, and corpus annotation and annotation tools. We believe that jointly this collection of chapters provides a good picture of how far we are today within discourse and dialogue and of important ch- lenges ahead. On this background we hope that computer scientists, engineers, and others who work in the broad area of discourse and dialogue, no matter if from an academic or industrial perspective, may bene?t from the book and ?nd it useful to their own work. Graduate students and Ph.D. students focusing on topics in discourse and dialogue may also ?nd the book interesting and pro?t from reading it. This book edition is based on a selected subset of papers from the successful 6th SIGdial Workshop on Discourse and Dialogue held in Lisbon, Portugal, in September 2005 in conjunction with the 9th Eurospeech (Interspeech) conf- ence. SIGdial is a special interest group on discourse and dialogue sponsored jointly by the two parent organisations ACL (Association for Computational Linguistics) and ISCA (International Speech Communication Association).

Speech and Human-Machine Dialog focuses on the dialog management component of a spoken language dialog system. Spoken language dialog systems provide a natural interface between humans and computers. These systems are of special interest for interactive applications, and they integrate several technologies including speech recognition, natural language understanding, dialog management and speech synthesis. Due to the conjunction of several factors throughout the past few years, humans are significantly changing their behavior vis-a-vis machines. In particular, the use of speech technologies will become normal in the professional domain, and in everyday life. The performance of speech recognition components has also significantly improved. This book includes various examples that illustrate the different functionalities of the dialog model in a representative application for train travel information retrieval (train time tables, prices and ticket reservation). Speech and Human-Machine Dialog is designed for a professional audience, composed of researchers and practitioners in industry.This book is also suitable as a secondary text for graduate-level students in computer science and engineering.

The development of computer and telecommunication technologies led to a revolutioninthewaythatpeopleworkandcommunicatewitheachother.One of the results is that large amount of information will increasingly be held in a form that is natural for users, as speech in natural language. In the presented work, we investigate the speech signal capture problem, which includes the separation of multiple interfering speakers using microphone arrays. Adaptive beamforming is a classical approach which has been developed since the seventies. However it requires a double-talk detector (DTD) that interrupts the adaptation when the target is active, since otherwise target cancelation occurs. The fact that several speakers may be active simulta- ouslymakesthisdetectiondi?cult,andifadditionalbackgroundnoiseoccurs, even less reliable. Our proposed approaches address this separation problem using continuous, uninterrupted adaptive algorithms. The advantage seems twofold:Firstly,thealgorithmdevelopmentismuchsimplersincenodetection mechanism needs to be designed and no threshold is to be tuned. Secondly, the performance may be improved due to the adaptation during periods of double-talk. In the ?rst part of the book, we investigate a modi?cation of the widely usedNLMSalgorithm,termedImplicitLMS(ILMS),whichimplicitlyincludes an adaptation control and does not require any threshold. Experimental ev- uations reveal that ILMS mitigates the target signal cancelation substantially with the distributed microphone array. However, in the more di?cult case of the compact microphone array, this algorithm does not su?ciently reduce the target signal cancelation. In this case, more sophisticated blind source se- ration techniques (BSS) seem necessary.

Bandwidth Extension of Speech Signals describes the theory and methods for quality enhancement of clean speech signals and distorted speech signals such as those that have undergone a band limitation, for instance, in a telephone network. Problems and the respective solutions are discussed for the different approaches. The different approaches are evaluated and a real-time implementation of the most promising approach is presented. The book includes topics related to speech coding, pattern- / speech recognition, speech enhancement, statistics and digital signal processing in general.

Over the last decade a number of research areas have contributed to the concept of advanced intelligent environments, these include ubiquitous computing, pervasive computing, embedded intelligence, intelligent user interfaces, human factors, intelligent buildings, mobile communications, domestic robots, intelligent sensors, artistic and architectural design and ambient intelligence. Undeniably, multimodal spoken language dialogue interaction is a key factor in ensuring natural interaction and therefore of particular interest for advanced intelligent environments. The book covers all the key topics in the field of intelligent environments from a variety of leading researchers.

Incorporating Knowledge Sources into Statistical Speech Recognition addresses the problem of developing efficient automatic speech recognition (ASR) systems, which maintain a balance between utilizing a wide knowledge of speech variability, while keeping the training / recognition effort feasible and improving speech recognition performance. The book provides an efficient general framework to incorporate additional knowledge sources into state-of-the-art statistical ASR systems. It can be applied to many existing ASR problems with their respective model-based likelihood functions in flexible ways.

Proactive Spoken Dialogue Interaction in Multi-Party Environments describes spoken dialogue systems that act as independent dialogue partners in the conversation with and between users. The resulting novel characteristics such as proactiveness and multi-party capabilities pose new challenges on the dialogue management component of such a system and require the use and administration of an extensive dialogue history. In order to assist the proactive spoken dialogue systems development, a comprehensive data collection seems mandatory and may be performed in a Wizard-of-Oz environment. Such an environment builds also the appropriate basis for an extensive usability and acceptance evaluation.Proactive Spoken Dialogue Interaction in Multi-Party Environments is a useful reference for students and researchers in speech processing.

Spoken Dialogue Systems Technology and Design covers key topics in the field of spoken language dialogue interaction from a variety of leading researchers. It brings together several perspectives in the areas of corpus annotation and analysis, dialogue system construction, as well as theoretical perspectives on communicative intention, context-based generation, and modelling of discourse structure. These topics are all part of the general research and development within the area of discourse and dialogue with an emphasis on dialogue systems; corpora and corpus tools and semantic and pragmatic modelling of discourse and dialogue.

Adaptive Multimodal Interactive Systems introduces a general framework for adapting multimodal interactive systems and comprises a detailed discussion of each of the steps required for adaptation. This book also investigates how interactive systems may be improved in terms of usability and user friendliness while describing the exhaustive user tests employed to evaluate the presented approaches.After introducing general theory, a generic approach for user modeling in interactive systems is presented, ranging from an observation of basic events to a description of higher-level user behavior. Adaptations are presented as a set of patterns similar to those known from software or usability engineering.These patterns describe recurring problems and present proven solutions. The authors include a discussion on when and how to employ patterns and provide guidance to the system designer who wants to add adaptivity to interactive systems. In addition to these patterns, the book introduces an adaptation framework, which exhibits an abstraction layer using Semantic Web technology.Adaptations are implemented on top of this abstraction layer by creating a semantic representation of the adaptation patterns. The patterns cover both graphical interfaces as well as speech-based and multimodal interactive systems.

Reasoning for Information: Seeking and Planning Dialogues provides a logic-based reasoning component for spoken language dialogue systems. This component, called Problem Assistant is responsible for processing constraints on a possible solution obtained from various sources, namely user and the system's domain-specific information. The authors also present findings on the implementation of a dialogue management interface to the Problem Assistant. The dialogue system supports simple mixed-initiative planning interactions in the TRAINS domain, which is still a relatively complex domain involving a number of logical constraints and relations forming the basis for the collaborative problem-solving behavior that drives the dialogue.

Stochastically-Based Semantic Analysis investigates the problem of automatic natural language understanding in a spoken language dialog system. The focus is on the design of a stochastic parser and its evaluation with respect to a conventional rule-based method. Stochastically-Based Semantic Analysis will be of most interest to researchers in artificial intelligence, especially those in natural language processing, computational linguistics, and speech recognition. It will also appeal to practicing engineers who work in the area of interactive speech systems.

Novel Techniques for Dialectal Arabic Speech describes approaches to improve automatic speech recognition for dialectal Arabic. Since speech resources for dialectal Arabic speech recognition are very sparse, the authors describe how existing Modern Standard Arabic (MSA) speech data can be applied to dialectal Arabic speech recognition, while assuming that MSA is always a second language for all Arabic speakers. In this book, Egyptian Colloquial Arabic (ECA) has been chosen as a typical Arabic dialect. ECA is the first ranked Arabic dialect in terms of number of speakers, and a high quality ECA speech corpus with accurate phonetic transcription has been collected. MSA acoustic models were trained using news broadcast speech. In order to cross-lingually use MSA in dialectal Arabic speech recognition, the authors have normalized the phoneme sets for MSA and ECA. After this normalization, they have applied state-of-the-art acoustic model adaptation techniques like Maximum Likelihood Linear Regression (MLLR) and Maximum A-Posteriori (MAP) to adapt existing phonemic MSA acoustic models with a small amount of dialectal ECA speech data. Speech recognition results indicate a significant increase in recognition accuracy compared to a baseline model trained with only ECA data.

In Monitoring Adaptive Spoken Dialog Systems, authors Alexander Schmitt and Wolfgang Minker investigate statistical approaches that allow for recognition of negative dialog patterns in Spoken Dialog Systems (SDS). The presented stochastic methods allow a flexible, portable and  accurate use. Beginning with the foundations of machine learning and pattern recognition, this monograph examines how frequently users show negative emotions in spoken dialog systems and develop novel approaches to speech-based emotion recognition using hybrid approach to model emotions. The authors make use of statistical methods based on acoustic, linguistic and contextual features to examine the relationship between the interaction flow and the occurrence of emotions using non-acted  recordings several thousand real users from commercial and non-commercial SDS.Additionally, the authors present novel statistical methods that spot problems within a dialog based on interaction patterns. The approaches enable future SDS to offer more natural and robust interactions. This work provides insights, lessons and  inspiration for future research and development, not only for spoken dialog systems, but for data-driven approaches to human-machine interaction in general.

Introducing Spoken Dialogue Systems into Intelligent Environments outlines the formalisms of a novel knowledge-driven framework for spoken dialogue management and presents the implementation of a model-based Adaptive Spoken Dialogue Manager(ASDM) called OwlSpeak. The authors have identified three stakeholders that potentially influence the behavior of the ASDM: the user, the SDS, and a complex Intelligent Environment (IE) consisting of various devices, services, and task descriptions. The theoretical foundation of a working ontology-based spoken dialogue description framework, the prototype implementation of the ASDM, and the evaluation activities that are presented as part of this book contribute to the ongoing spoken dialogue research by establishing the fertile ground of model-based adaptive spoken dialogue management. This monograph is ideal for advanced undergraduate students, PhD students, and postdocs as well as academic and industrial researchers and developers in speech and multimodal interactive systems.

Speech and Human-Machine Dialog focuses on the dialog management component of a spoken language dialog system. Spoken language dialog systems provide a natural interface between humans and computers. These systems are of special interest for interactive applications, and they integrate several technologies including speech recognition, natural language understanding, dialog management and speech synthesis. Due to the conjunction of several factors throughout the past few years, humans are significantly changing their behavior vis-à-vis machines. In particular, the use of speech technologies will become normal in the professional domain, and in everyday life. The performance of speech recognition components has also significantly improved. This book includes various examples that illustrate the different functionalities of the dialog model in a representative application for train travel information retrieval (train time tables, prices and ticket reservation). Speech and Human-Machine Dialog is designed for a professional audience, composed of researchers and practitioners in industry. This book is also suitable as a secondary text for graduate-level students in computer science and engineering.

Spoken Dialogue Systems Technology and Design covers key topics in the field of spoken language dialogue interaction from a variety of leading researchers. It brings together several perspectives in the areas of corpus annotation and analysis, dialogue system construction, as well as theoretical perspectives on communicative intention, context-based generation, and modelling of discourse structure. These topics are all part of the general research and development within the area of discourse and dialogue with an emphasis on dialogue systems; corpora and corpus tools and semantic and pragmatic modelling of discourse and dialogue.

Proactive Spoken Dialogue Interaction in Multi-Party Environments describes spoken dialogue systems that act as independent dialogue partners in the conversation with and between users. The resulting novel characteristics such as proactiveness and multi-party capabilities pose new challenges on the dialogue management component of such a system and require the use and administration of an extensive dialogue history. In order to assist the proactive spoken dialogue systems development, a comprehensive data collection seems mandatory and may be performed in a Wizard-of-Oz environment. Such an environment builds also the appropriate basis for an extensive usability and acceptance evaluation.Proactive Spoken Dialogue Interaction in Multi-Party Environments is a useful reference for students and researchers in speech processing.

Adaptive Multimodal Interactive Systems introduces a general framework for adapting multimodal interactive systems and comprises a detailed discussion of each of the steps required for adaptation. This book also investigates how interactive systems may be improved in terms of usability and user friendliness while describing the exhaustive user tests employed to evaluate the presented approaches.After introducing general theory, a generic approach for user modeling in interactive systems is presented, ranging from an observation of basic events to a description of higher-level user behavior. Adaptations are presented as a set of patterns similar to those known from software or usability engineering.These patterns describe recurring problems and present proven solutions. The authors include a discussion on when and how to employ patterns and provide guidance to the system designer who wants to add adaptivity to interactive systems. In addition to these patterns, the book introduces an adaptation framework, which exhibits an abstraction layer using Semantic Web technology.Adaptations are implemented on top of this abstraction layer by creating a semantic representation of the adaptation patterns. The patterns cover both graphical interfaces as well as speech-based and multimodal interactive systems.

Intelligent environments represent an emerging topic in research. Next Generation Intelligent Environments: Ambient Adaptive Systems will cover all key topics in the field of intelligent ambient adaptive systems. It focuses on the results worked out within the framework of the ATRACO (Adaptive and TRusted Ambient eCOlogies) project. The theoretical background, the developed prototypes, and the evaluated results form a fertile ground useful for the broad intelligent environments scientific community as well as for industrial interest groups. Features of the book include:A unique and original collection of chapters on intelligent ambient adaptive systemsBroad coverage of the field of intelligent environments research and evaluation, as well as topics such as adaptation within activity spheresDeveloped prototypes as examples for readersComputer scientists, engineers and others who work in the area of ambient environments will find the edition interesting and useful to their own work. In addition, graduate students and Ph.D. students specializing in the area of intelligent environments may also use this book to get a concrete idea of the major issues to consider when developing intelligent environments in practice.

Reasoning for Information: Seeking and Planning Dialogues provides a logic-based reasoning component for spoken language dialogue systems. This component, called Problem Assistant is responsible for processing constraints on a possible solution obtained from various sources, namely user and the system's domain-specific information. The authors also present findings on the implementation of a dialogue management interface to the Problem Assistant. The dialogue system supports simple mixed-initiative planning interactions in the TRAINS domain, which is still a relatively complex domain involving a number of logical constraints and relations forming the basis for the collaborative problem-solving behavior that drives the dialogue.

In Monitoring Adaptive Spoken Dialog Systems, authors Alexander Schmitt and Wolfgang Minker investigate statistical approaches that allow for recognition of negative dialog patterns in Spoken Dialog Systems (SDS). The presented stochastic methods allow a flexible, portable and  accurate use. Beginning with the foundations of machine learning and pattern recognition, this monograph examines how frequently users show negative emotions in spoken dialog systems and develop novel approaches to speech-based emotion recognition using hybrid approach to model emotions. The authors make use of statistical methods based on acoustic, linguistic and contextual features to examine the relationship between the interaction flow and the occurrence of emotions using non-acted  recordings several thousand real users from commercial and non-commercial SDS.Additionally, the authors present novel statistical methods that spot problems within a dialog based on interaction patterns. The approaches enable future SDS to offer more natural and robust interactions. This work provides insights, lessons and  inspiration for future research and development, not only for spoken dialog systems, but for data-driven approaches to human-machine interaction in general.