Deep Learning and Linguistic Representation

Shalom Lappin is Professor of Natural Language Processing at Queen Mary University of London, Professor of Computational Linguistics at the University of Gothenburg and Emeritus Professor of Computational Linguistics at King’s College London.

This book is a very timely synthesis of classical linguistics that the author has worked in for several decades and the modern revolution in NLP enabled by Deep Learning. It also asks provocative foundational questions about whether traditional grammars are the most suitable representations of linguistic structure or if we need to go beyond them.-- Devdatt Dubhashi, Professor, Chalmers UniversityDeep neural networks are having a tremendous impact on applied natural language processing. In this clearly written book, Shalom Lappin tackles the novel and exciting question of what are their implications for theories of language acquisition, representation and usage. This will be an enlightening reading for anybody interested in language from the perspectives of theoretical linguistics, cognitive science, AI and the philosophy of science.-- Marco Baroni, ICREA Research Professor, Facebook AI Research Scientist

• Chapter 1 Introduction: Deep Learning in Natural Language Processing 1.1 OUTLINE OF THE BOOK 1.2 FROM ENGINEERING TO COGNITIVE SCIENCE 1.3 ELEMENTS OF DEEP LEARNING 1.4 TYPES OF DEEP NEURAL NETWORKS 1.5 AN EXAMPLE APPLICATION 1.6 SUMMARY AND CONCLUSIONS Chapter 2 Learning Syntactic Structure with Deep Neural Networks 2.1 SUBJECT-VERB AGREEMENT 2.2 ARCHITECTURE AND EXPERIMENTS 2.3 HIERARCHICAL STRUCTURE 2.4 TREE DNNS 2.5 SUMMARY AND CONCLUSIONS Chapter 3 Machine Learning and The Sentence Acceptability Task 3.1 GRADIENCE IN SENTENCE ACCEPTABILITY 3.2 PREDICTING ACCEPTABILITY WITH MACHINE LEARNING MODELS 3.3 ADDING TAGS AND TREES 3.4 SUMMARY AND CONCLUSIONS Chapter 4 Predicting Human Acceptability Judgments in Context4.1 ACCEPTABILITY JUDGMENTS IN CONTEXT 4.2 TWO SETS OF EXPERIMENTS 4.3 THE COMPRESSION EFFECT AND DISCOURSE COHERENCE4.4 PREDICTING ACCEPTABILITY WITH DIFFERENT DNN MODELS 4.5 SUMMARY AND CONCLUSIONS Chapter 5 Cognitively Viable Computational Models of Linguistic Knowledge 5.1 HOW USEFUL ARE LINGUISTIC THEORIES FOR NLP APPLICATIONS? 5.2 MACHINE LEARNING MODELS VS FORMAL GRAMMAR5.3 EXPLAINING LANGUAGE ACQUISITION 5.4 DEEP LEARNING AND DISTRIBUTIONAL SEMANTICS 15.5 SUMMARY AND CONCLUSIONS Chapter 6 Conclusions and Future Work 6.1 REPRESENTING SYNTACTIC AND SEMANTIC KNOWLEDGE6.2 DOMAIN SPECIFIC LEARNING BIASES AND LANGUAGE ACQUISITION 6.3 DIRECTIONS FOR FUTURE WORK REFERENCES Author IndexSubject Index