Text, Speech and Language Technology – serie

Corpus-based methods will be found at the heart of many language and speech processing systems. This book provides an in-depth introduction to these technologies through chapters describing basic statistical modeling techniques for language and speech, the use of Hidden Markov Models in continuous speech recognition, the development of dialogue systems, part-of-speech tagging and partial parsing, data-oriented parsing and n-gram language modeling. The book attempts to give both a clear overview of the main technologies used in language and speech processing, along with sufficient mathematics to understand the underlying principles. There is also an extensive bibliography to enable topics of interest to be pursued further. Overall, we believe that the book will give newcomers a solid introduction to the field and it will give existing practitioners a concise review of the principal technologies used in state-of-the-art language and speech processing systems. Corpus-Based Methods in Language and Speech Processing is an initiative of ELSNET, the European Network in Language and Speech. In its activities, ELSNET attaches great importance to the integration of language and speech, both in research and in education. The need for and the potential of this integration are well demonstrated by this publication.

An Introduction to Text-to-Speech Synthesis is a comprehensive introduction to the subject. The author treats two areas of speech synthesis: Part I of the book concerns natural language processing and the inherent problems it presents for speech synthesis; Part II focuses on digital signal processing, with an emphasis on the concatenative approach. Both parts of the text guide the reader through the material in a step-by-step easy-to-follow way. This is the first book to treat the topic of speech synthesis from the perspective of two different engineering approaches. The book will be of interest to researchers and students in phonetics and speech communication, in both academia and industry.

Researchers in a number of disciplines deal with large text sets requiring both text management and text analysis. Faced with a large amount of textual data collected in marketing surveys, literary investigations, historical archives and documentary data bases, these researchers require assistance with organizing, describing and comparing texts. This volume demonstrates how exploratory multivariate statistical methods such as correspondence analysis and cluster analysis can be used to help investigate, assimilate and evaluate textual data. The main text does not contain any strictly mathematical demonstrations, making it accessible to a large audience. Full definitions of concepts, implementations of procedures and rules for reading and interpreting results are fully explored. A succession of examples is intended to allow the reader to appreciate the variety of actual and potential applications and the complementary processing methods.

This book is a revised version of my doctoral thesis which was submitted in April 1993. The main extension is a chapter on evaluation of the system de­ scribed in Chapter 8 as this is clearly an issue which was not treated in the original version. This required the collection of data, the development of a concept for diagnostic evaluation of linguistic word recognition systems and, of course, the actual evaluation of the system itself. The revisions made primarily concern the presentation of the latest version of the SILPA system described in an additional Subsection 8. 3, the development environment for SILPA in Sec­ tion 8. 4, the diagnostic evaluation of the system as an additional Chapter 9. Some updates are included in the discussion of phonology and computation in Chapter 2 and finite state techniques in computational phonology in Chapter 3. The thesis was designed primarily as a contribution to the area of compu­ tational phonology. However, it addresses issues which are relevant within the disciplines of general linguistics, computational linguistics and, in particular, speech technology, in providing a detailed declarative, computationally inter­ preted linguistic model for application in spoken language processing. Time Map Phonology is a novel, constraint-based approach based on a two-stage temporal interpretation of phonological categories as events.

Via a number of articles, this book presents a survey and a set of projects in computational lexical semantics. The most crucial aspects of ongoing research on predicates are presented: verb semantic classifications, relations between syntax and semantics, Wordnet for verbs, multilinguism, lexical knowledge bases and lexical acquisition, the generative lexicon. Predicative forms in natural language and in lexical knowledge basis is designed for professors, researchers and graduate students in the area of language processing and semantics.

The last decade has been one of dramatic progress in the field of Natural Language Processing (NLP). This hitherto largely academic discipline has found itself at the center of an information revolution ushered in by the Internet age, as demand for human-computer communication and informa­ tion access has exploded. Emerging applications in computer-assisted infor­ mation production and dissemination, automated understanding of news, understanding of spoken language, and processing of foreign languages have given impetus to research that resulted in a new generation of robust tools, systems, and commercial products. Well-positioned government research funding, particularly in the U. S. , has helped to advance the state-of-the­ art at an unprecedented pace, in no small measure thanks to the rigorous 1 evaluations. This volume focuses on the use of Natural Language Processing in In­ formation Retrieval (IR), an area of science and technology that deals with cataloging, categorization, classification, and search of large amounts of information, particularly in textual form. An outcome of an information retrieval process is usually a set of documents containing information on a given topic, and may consist of newspaper-like articles, memos, reports of any kind, entire books, as well as annotated image and sound files. Since we assume that the information is primarily encoded as text, IR is also a natural language processing problem: in order to decide if a document is relevant to a given information need, one needs to be able to understand its content.

This text provides an introduction to the acoustic analysis and characteristics of speech sounds. The first part of the book covers aspects of the source-filter decomposition of speech, spectrographic analysis, the acoustic theory of speech production and acoustic phonetic cues. The second part is based on computational techniques for analyzing the acoustic speech signal including digital time and frequency analyzes, formant synthesis, and the linear predictive coding of speech. There is also an introductory chapter on the classification of acoustic speech signals which is relevant to aspects of automatic speech and talker recognition. Included with the book is a CD-ROM containing extensive speech corpora, the EMU speech analysis tools, extensions to the X-LISP-STAT programming language that are adapted to speech analysis, and numerous exercises that are linked to the major themes of the book and which can be run on Windows-95 and UNIX platforms.The book and CD-ROM are intended for use as teaching materials on undergraduate and postgraduate speech acoustics and experimental phonetics courses; they are also aimed at researchers from phonetics, linguistics, computer science, psychology and engineering who wish to gain an understanding of the basis of speech acoustics and its application to fields such as speech synthesis and automatic speech recognition.

This introduction to the acoustic analysis and characteristics of speech sounds covers aspects of the source-filter decomposition of speech, spectrographic analysis, the acoustic theory of speech production and acoustic phonetic cues. The second part is based on computational techniques for analysing the acoustic speech signal including digital time and frequency analyses, formant synthesis, and the linear predictive coding of speech. There is also an introductory chapter on the classification of acoustic speech signals which is relevant to aspects of automatic speech and talker recognition. Included with the book is a CD-ROM containing extensive speech corpora, the EMU speech analysis tools, extensions to the X-LISP-STAT programming language that are adapted to speech analysis, and numerous exercises that are linked to the major themes of the book and which can be run on Windows-95 and UNIX platforms.The book and CD-ROM are intended for use as teaching materials on undergraduate and postgraduate speech acoustics and experimental phonetics courses; they are also aimed at researchers from phonetics, linguistics, computer science, psychology and engineering who wish to gain an understanding of the basis of speech acoustics and its application to fields such as speech synthesis and automatic speech recognition.

This work provides an in-depth discussion of the field of syntactic wordclass tagging - the annotation of the words in a text with tags indicating their syntactic properties. Represented are the viewpoints of the two main groups who take an interest in tagging: the users of tagged text and the developers of tagging software. The book starts out by examining the field from the user's point of view. After a brief historical overview, the nature and uses of tagging are discussed and current practice is described. Here the user will find what tagging is and the software developer what it is the user wants. The book then switches to the other point of view and continues with a detailed explanation of the most common computational techniques for automatically tagging large amounts of text. Here the software developer finds information needed for the implementation of a tagger while the user gains insight into the possibilities and impossibilities of automatic tagging and how computer-provided tags should be interpreted.

Most books about computational (lexical) semantic lexicons deal with the depth (or content) aspect of lexicons, ignoring the breadth (or coverage) aspect. This book presents an attempt to address both issues: content and coverage of computational semantic lexicons. Moreover, it addresses issues in implemented systems such as the application time of lexical rules. Lexical rules and lexical underspecification are also contrasted in implemented systems. The main approaches in the field of computational (lexical) semantics are represented in the present book (including Wordnet, CyC, Mikrokosmos, Generative Lexicon). This book embraces several fields (and subfields) as different as: linguistics (theoretical, computational, semantics, pragmatics), psycholinguistics, cognitive science, computer science, artificial intelligence, knowledge representation, statistics and natural language processing.

The 1990s have been an interesting time for researchers working with large collections of text. It was not all that long ago that researchers referred to the Brown Corpus as a `large' corpus. The Brown Corpus, a `mere' million words collected at Brown University in the 1960s, is about the same size as a dozen novels, the complete works of William Shakespeare, the Bible, a collegiate dictionary or a week of a newswire service. Today, one can easily surf the web and download millions of words in no time at all. What can we do with all this data? It is better to do something simple than nothing at all. Researchers in large corpora are using basically brute force methods to make progress on some of the hardest problems in natural language processing, including part-of-speech tagging, word sense disambiguation, parsing, machine translation, information retrieval and discourse analysis.They are overcoming the so-called knowledge-acquisition bottleneck by processing vast quantities of data, more text than anyone could possibly read in a lifetime, and estimating all sorts of `central and typical' facts that any speaker of the language would be expected to know, for example, word frequencies, word associations and typical predicate-argument relations. Much of this work has been reported at a series of annual meetings, known as the Workshop on Very Large Corpora (WVLC) and related meetings sponsored by ACL/SIGDAT (Association for Computational Linguistics' special interest group on data). Subsequent meetings have been held in Asia (1994, 1997), America (1995, 1996, 1997) and Europe (1995, 1996). The papers in this book represent much of the best of the first three years of this workshop/conference as selected by a competitive review process. From the foreword: `The bipolar transistor has a remarkable characteristic that makes it unique as a circuit design element; it displays an exponential relationship between collector current and base-to-emitter voltage that is highly accurate over an extremely wide range of currents.This conformance to a mathematical law opens up numerous possibilities for analog signal processing. Log filters represent one of the most interesting applications of this exponential relationship.' Robert Adams, Analog Devices.

This work offers a state-of-the-art survey of methods and techniques for structuring, acquiring and maintaining lexical resources for speech and language processing. The first chapter provides a broad survey of the field of computational lexicography, introducing most of the issues, terms and topics which are addressed in more detail in the rest of the book. The last chapter takes a psycholinguistic perspective and addresses the relation between storage and computation in the mental lexicon. The relevance of these topics for speech and language processing is obvious, for since NLP systems need large lexica in order to achieve reasonable coverage, and since the construction and maintenance of large-size lexical resources is a complex and costly task, it is of crucial importance for those who design or build such systems to be aware of the latest developments in this fast-moving field.

This work offers a survey of methods and techniques for structuring, acquiring and maintaining lexical resources for speech and language processing. The first chapter provides a broad survey of the field of computational lexicography, introducing most of the issues, terms and topics which are addressed in more detail in the rest of the book. The next two chapters focus on the structure and the content of man-made lexicons, concentrating respectively on (morpho-)syntactic and (morpho-)phonological information. Both chapters adopt a declarative constraint-based methodology and pay ample attention to the various ways in which lexical generalizations can be formalized and exploited to enhance the consistency and to reduce the redundancy of lexicons. A complementary perspective is offered in the next two chapters, which present techniques for automatically deriving lexical resources from text corpora. These chapters adopt an inductive data-oriented methodology and focus also on methods for tokenization, lemmatization and shallow parsing. The next three chapters focus on speech synthesis and speech recognition.The last chapter takes a psycholinguistic perspective and addresses the relation between storage and computation in the mental lexicon. The work should be helpful for advanced students and professional scientists working in the areas of computational linguistics and language and speech technology.

With the rising importance of multilingualism in language industries, brought about by global markets and world-wide information exchange, parallel corpora, that is, corpora of texts accompanied by their translation, have become key resources in the development of natural language processing tools. The applications based upon parallel corpora are numerous and growing in number: for example, multilingual lexicography and terminology, machine and human translation, cross-language information retrieval, and language learning. The book's chapters have been commissioned from major figures in the field of parallel corpus building and exploitation, with the aim of showing the state-of-the art in parallel text alignment.

The study of prosody has undergone noticeable development during the years of the 20th century. This volume contains contributions by 15 internationally recognized experts in different areas of prosody which provide readers with the most current and comprehensive picture of the major areas of research within the field. The contributions not only provide a survey of major developments in prosody during the past 15-20 years, but also present ongoing work as well as point to areas where future research is needed. The chapters deal with a wide range of topics including the representation of tones and intonation, evidence for and constraints on prosodic phrasing, prosodic boundary detection, articulatory dynamics of stress, timing in speech, and prosodic correlates of speaking style, as well as the perception of prosodic prominence.

This volume covers the main aspects of intonation, written by international researchers in the field. Following the introduction, fourteen chapters are organized into five thematic sections: overview of intonation, prominence and focus, boundaries and discourse, intonation modelling and intonation technology. Each chapter is basically autonomous within a thematic section, but the subject of several chapters extends over more than one thematic section. The combination of a wide range of research areas, as well as interdisciplinary approaches in the study of intonation, makes this volume a unique contribution to the international scientific community. Basic knowledge of intonation and prosody is assumed in the context of linguistic and computational backgrounds.

Parsing technology is concerned with finding syntactic structure in language. In parsing we have to deal with incomplete and not necessarily accurate formal descriptions of natural languages. Robustness and efficiency are among the main issuesin parsing. Corpora can be used to obtain frequency information about language use. This allows probabilistic parsing, an approach that aims at both robustness and efficiency increase. Approximation techniques, to be applied at the level of language description, parsing strategy, and syntactic representation, have the same objective. Approximation at the level of syntactic representation is also known as underspecification, a traditional technique to deal with syntactic ambiguity. In this book new parsing technologies are collected that aim at attacking the problems of robustness and efficiency by exactly these techniques: the design of probabilistic grammars and efficient probabilistic parsing algorithms, approximation techniques applied to grammars and parsers to increase parsing efficiency, and techniques for underspecification and the integration of semantic information in the syntactic analysis to deal with massive ambiguity. The book gives a state-of-the-art overview of current research and development in parsing technologies. In its chapters we see how probabilistic methods have entered the toolbox of computational linguistics in order to be applied in both parsing theory and parsing practice. The book is both a unique reference for researchers and an introduction to the field for interested graduate students.

This volume covers the main aspects of intonation, written by international researchers in the field. Following the introduction, fourteen chapters are organized into five thematic sections: overview of intonation, prominence and focus, boundaries and discourse, intonation modelling and intonation technology. Each chapter is basically autonomous within a thematic section, but the subject of several chapters extends over more than one thematic section. The combination of a wide range of research areas, as well as interdisciplinary approaches in the study of intonation, makes this volume a unique contribution to the international scientific community. Basic knowledge of intonation and prosody is assumed in the context of linguistic and computational backgrounds.

In this book we address robustness issues at the speech recognition and natural language parsing levels, with a focus on feature extraction and noise robust recognition, adaptive systems, language modeling, parsing, and natural language understanding. This book attempts to give a clear overview of the main technologies used in language and speech processing, along with an extensive bibliography to enable topics of interest to be pursued further. It also brings together speech and language technologies often considered separately. Robustness in Language and Speech Technology serves as a valuable reference and although not intended as a formal university textbook, contains some material that can be used for a course at the graduate or undergraduate level.

This work is a comprehensive introduction to the statistical analysis of word frequency distributions, intended for computational linguists, corpus linguists, psycholinguists, and researchers in the field of quantitative stylistics. Word frequency distributions are characterized by very large numbers of rare words. This property leads to strange phenomena such as mean frequencies that systematically change as the number of observations is increased, relative frequencies that even in large samples are not fully reliable estimators of population probabilities, and model parameters that vary with text or corpus size. Special statistical techniques for the analysis of distributions with large numbers of rare events can be found in various technical journals. The aim of this book is to make these techniques more accessible for non-specialists, both theoretically, by means of a careful introduction to the underlying probabilistic and statistical concepts, and practically, by providing a program library implementing the main models for word frequency distributions.

An Introduction to Text-to-Speech Synthesis is a comprehensive introduction to the subject. The author treats two areas of speech synthesis: Part I of the book concerns natural language processing and the inherent problems it presents for speech synthesis; Part II focuses on digital signal processing, with an emphasis on the concatenative approach. Both parts of the text guide the reader through the material in a step-by-step easy-to-follow way. This is the first book to treat the topic of speech synthesis from the perspective of two different engineering approaches. The book will be of interest to researchers and students in phonetics and speech communication, in both academia and industry.

In Marcus (1980), deterministic parsers were introduced. These are parsers which satisfy the conditions of Marcus's determinism hypothesis, i.e., they are strongly deterministic in the sense that they do not simulate non­ determinism in any way. In later work (Marcus et al. 1983) these parsers were modified to construct descriptions of trees rather than the trees them­ selves. The resulting D-theory parsers, by working with these descriptions, are capable of capturing a certain amount of ambiguity in the structures they build. In this context, it is not clear what it means for a parser to meet the conditions of the determinism hypothesis. The object of this work is to clarify this and other issues pertaining to D-theory parsers and to provide a framework within which these issues can be examined formally. Thus we have a very narrow scope. We make no ar­ guments about the linguistic issues D-theory parsers are meant to address, their relation to other parsing formalisms or the notion of determinism in general. Rather we focus on issues internal to D-theory parsers themselves.

This work covers the topic of multimodality from a large number of different perspectives and provides the advanced student/researcher with a survey of theories of multimodal communication between people as well as reviewing many aspects of multimodal input/output in technical systems. Chapters dealing with human-human multimodal communication include speech-gesture systems, semiotics of gesture, structure and functions of face-to-face communication, emotional relations and intercultural variation, and human-human communication which is mediated by computer for the handicapped. Chapters dealing with human-machine communication and interfaces cover the technology and science of creating talking faces, technology and methods for the development of animated interface agents in intelligent multimedia systems, and the integration of multimodal input and output in the computer interface. The book also covers computer processing and understanding of signal and symbol input from speech, text, and visual images.

A comprehensive introduction to the statistical analysis of word frequency distributions, this text is intended for computational linguists, corpus linguists, psycholinguists, and researchers in the field of quantitative stylistics. Word frequency distributions are characterized by very large numbers of rare words. This property leads to strange phenomena such as mean frequencies that systematically change as the number of observations is increased, relative frequencies that even in large samples are not fully reliable estimators of population probabilities, and model parameters that vary with text or corpus size. Special statistical techniques for the analysis of distributions with large numbers of rare events can be found in various technical journals. The aim of this book is to make these techniques more accessible for non-specialists, both theoretically, by means of a careful introduction to the underlying probabilistic and statistical concepts, and practically, by providing a programme library implementing the main models for word frequency distributions.

Linguists and engineers in Natural Language Processing tend to use electronic corpora more and more. Most research has long been limited to raw (unannotated) texts or to tagged texts (annotated with parts of speech only), but these approaches suffer from a word by word perspective. A new line of research involves corpora with richer annotations such as clauses and major constituents, grammatical functions and dependency links. The first parsed corpora were the English Lancaster treebank and Penn Treebank. New ones have recently been developed for other languages. This book + provides a state of the art on work being done with parsed corpora; + gathers 21 papers on building and using parsed corpora raising many relevant questions; + deals with a variety of languages and a variety of corpora; + is for those working in linguistics, computational linguistics, natural language, syntax, and grammar.

Linguists and engineers in natural language processing tend to use electronic corpora more and more. Most research has long been limited to raw (unannotated) texts or to tagged texts (annotated with parts of speech only), but these approaches suffer from a word by word perspective. A new line of research involves corpora with richer annotations such as clauses and major constituents, grammatical functions and dependency links. The first parsed corpora were the English Lancaster treebank and Penn treebank. New ones have recently been developed for other languages. This text provides an update on work being done with parsed corpora. It presents 21 papers on building and using parsed corpora raising many relevant questions, and deals with a variety of languages and a variety of corpora. It is intended for those working in linguistics, computational linguistics, natural language, syntax, and grammar.

This work is of relevance to researchers and programme developers in the field of Machine Translation and especially Example-Based Machine Translation, bilingual text processing and cross-linguistic information retrieval. It is also of interest to translation technologists and localisation professionals. Recent Advances in Example-Based Machine Translation fills a void, because it is the first book to tackle the issue of EBMT in depth. It gives a state-of-the-art overview of EBMT techniques and provides a coherent structure in which all aspects of EBMT are embedded. Its contributions are written by long-standing researchers in the field of MT in general, and EBMT in particular. This book can be used in graduate-level courses in machine translation and statistical NLP.

This work is of relevance to researchers and programme developers in the field of Machine Translation and especially Example-Based Machine Translation, bilingual text processing and cross-linguistic information retrieval. It is also of interest to translation technologists and localisation professionals. Recent Advances in Example-Based Machine Translation fills a void, because it is the first book to tackle the issue of EBMT in depth. It gives a state-of-the-art overview of EBMT techniques and provides a coherent structure in which all aspects of EBMT are embedded. Its contributions are written by long-standing researchers in the field of MT in general, and EBMT in particular. This book can be used in graduate-level courses in machine translation and statistical NLP.

The origins of this book arise from the highly successful second SIGdial Workshop on Discourse and Dialogue that was held in September 2001 in con­ junction with Eurospeech 2001. The original workshop proceedings consisted of 29 papers selected from 57 submissions, an exceptionally high number of submissions for a two day workshop. This book includes extended versions of 12 papers originally presented at the workshop. In addition, 4 other invited papers on major themes in discourse and dialogue research are included. There are three main themes addressed by the papers in this collection: (1) corpus annotation and analysis; (2) method­ ologies for construction of dialogue systems; and (3) perspectives on various key theoretical issues including communicative intention, context-based gen­ eration, and modeling of discourse structure. However, because of the very nature of discourse and dialogue research that often requires researchers to tackle several issues in one piece of work, we have chosen to order the papers alphabetically by author rather than try to create artificial thematic sections. We believe this collection provides a concise yet reasonably comprehensive snapshot of major research themes in discourse and dialogue. We hope that readers will benefit greatly from this collection.