Deep Learning and Linguistic Representation

162 Pages·2021·11.228 MB·English

by  Shalom Lappin

Deep Learning
and Linguistic
Representation
Chapman & Hall/CRC Machine Learning & Pattern Recognition
Introduction to Machine Learning with Applications in Information Security
Mark Stamp
A First Course in Machine Learning
Simon Rogers, Mark Girolami
Statistical Reinforcement Learning: Modern Machine Learning Approaches
Masashi Sugiyama
Sparse Modeling: Theory, Algorithms, and Applications
Irina Rish, Genady Grabarnik
Computational Trust Models and Machine Learning
Xin Liu, Anwitaman Datta, Ee-Peng Lim
Regularization, Optimization, Kernels, and Support Vector Machines
Johan A.K. Suykens, Marco Signoretto, Andreas Argyriou
Machine Learning: An Algorithmic Perspective, Second Edition
Stephen Marsland
Bayesian Programming
Pierre Bessiere, Emmanuel Mazer, Juan Manuel Ahuactzin, Kamel Mekhnacha
Multilinear Subspace Learning: Dimensionality Reduction of Multidimensional
Data
Haiping Lu, Konstantinos N. Plataniotis, Anastasios Venetsanopoulos
Data Science and Machine Learning: Mathematical and Statistical Methods
Dirk P. Kroese, Zdravko Botev, Thomas Taimre, Radislav Vaisman
Deep Learning and Linguistic Representation
Shalom Lappin
For more information on this series please visit: https://www.routledge.com/Chapman--Hall-
CRC-Machine-Learning--Pattern-Recognition/book-series/CRCMACLEAPAT
Deep Learning
and Linguistic
Representation
Shalom Lappin
First edition published 2021
by CRC Press
6000 Broken Sound Parkway NW, Suite 300, Boca Raton, FL 33487-2742
and by CRC Press
2 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN
© 2021 Shalom Lappin
CRC Press is an imprint of Taylor & Francis Group, LLC
The right of Shalom Lappin to be identified as author of this work has been asserted by him in accor-
dance with sections 77 and 78 of the Copyright, Designs and Patents Act 1988.
Reasonable efforts have been made to publish reliable data and information, but the author and pub-
lisher cannot assume responsibility for the validity of all materials or the consequences of their use.
The authors and publishers have attempted to trace the copyright holders of all material reproduced
in this publication and apologize to copyright holders if permission to publish in this form has not
been obtained. If any copyright material has not been acknowledged please write and let us know so
we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced,
transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or
hereafter invented, including photocopying, microfilming, and recording, or in any information stor-
age or retrieval system, without written permission from the publishers.
For permission to photocopy or use material electronically from this work, access www.copyright.
com or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA
01923, 978-750-8400. For works that are not available on CCC please contact mpkbookspermis-
[email protected]
Trademark notice: Product or corporate names may be trademarks or registered trademarks and are
used only for identification and explanation without intent to infringe.
Library of Congress Cataloging-in-Publication Data
Names: Lappin, Shalom, author.
Title: Deep learning and linguistic representation / Shalom Lappin.
Description: Boca Raton : CRC Press, 2021. | Includes bibliographical
references and index.
Identifiers: LCCN 2020050622 | ISBN 9780367649470 (hardback) | ISBN
9780367648749 (paperback) | ISBN 9781003127086 (ebook)
Subjects: LCSH: Computational linguistics. | Natural language processing
(Computer science) | Machine learning.
Classification: LCC P98 .L37 2021 | DDC 410.285--dc23
LC record available at https://lccn.loc.gov/2020050622
ISBN: 978-0-367-64947-0 (hbk)
ISBN: 978-0-367-64874-9 (pbk)
ISBN: 978-1-003-12708-6 (ebk)
Typeset in Latin Modern font
by KnowledgeWorks Global Ltd.
הפש תבהא יהמ יתוא הדמילש ,ןיפל הדע ימא רכזל

Contents
Preface xi
Chapter1(cid:4)Introduction: Deep Learning in Natural Language
Processing 1
1.1 OUTLINE OF THE BOOK 1
1.2 FROM ENGINEERING TO COGNITIVE SCIENCE 4
1.3 ELEMENTS OF DEEP LEARNING 7
1.4 TYPES OF DEEP NEURAL NETWORKS 10
1.5 AN EXAMPLE APPLICATION 17
1.6 SUMMARY AND CONCLUSIONS 21
Chapter2(cid:4)Learning Syntactic Structure with Deep Neural
Networks 23
2.1 SUBJECT-VERB AGREEMENT 23
2.2 ARCHITECTURE AND EXPERIMENTS 24
2.3 HIERARCHICAL STRUCTURE 34
2.4 TREE DNNS 39
2.5 SUMMARY AND CONCLUSIONS 42
Chapter3(cid:4)Machine Learning and the Sentence Acceptability
Task 45
3.1 GRADIENCE IN SENTENCE ACCEPTABILITY 45
3.2 PREDICTINGACCEPTABILITYWITHMACHINELEARN-
ING MODELS 51
vii
viii (cid:4) Contents
3.3 ADDING TAGS AND TREES 62
3.4 SUMMARY AND CONCLUSIONS 66
Chapter4(cid:4)Predicting Human Acceptability Judgements in
Context 69
4.1 ACCEPTABILITY JUDGEMENTS IN CONTEXT 69
4.2 TWO SETS OF EXPERIMENTS 75
4.3 THE COMPRESSION EFFECT AND DISCOURSE CO-
HERENCE 78
4.4 PREDICTINGACCEPTABILITYWITHDIFFERENTDNN
MODELS 80
4.5 SUMMARY AND CONCLUSIONS 87
Chapter5(cid:4)Cognitively Viable Computational Models of Lin-
guistic Knowledge 89
5.1 HOW USEFUL ARE LINGUISTIC THEORIES FOR NLP
APPLICATIONS? 89
5.2 MACHINE LEARNING MODELS VS FORMAL GRAM-
MAR 92
5.3 EXPLAINING LANGUAGE ACQUISITION 96
5.4 DEEP LEARNING AND DISTRIBUTIONAL SEMANTICS 100
5.5 SUMMARY AND CONCLUSIONS 108
Chapter6(cid:4)Conclusions and Future Work 113
6.1 REPRESENTINGSYNTACTICANDSEMANTICKNOWL-
EDGE 113
6.2 DOMAIN-SPECIFIC LEARNING BIASES AND LAN-
GUAGE ACQUISITION 119
6.3 DIRECTIONS FOR FUTURE WORK 121
REFERENCES 123
Contents (cid:4) ix
Author Index 139
Subject Index 145