Table Of Content

MACHINE LEARNING 010001010111010001101000 011001010110110100100000 010000010110110001110000 011000010111100101100100 011010010110111000001101 000010100100110101100001 011000110110100001101001 011011100110010100100000 010011000110010101100001 011100100110111001101001 011011100110011100001101 000010100101010001101000 011001010010000001001110 011001010111011100100000 010000010100100100001101 00001010 The MIT Press Essential Knowledge Series Auctions, Timothy P. Hubbard and Harry J. Paarsch Cloud Computing, Nayan Ruparelia Computing: A Concise History, Paul E. Ceruzzi The Conscious Mind, Zoltan L. Torey Crowdsourcing, Daren C. Brabham Free Will, Mark Balaguer Information and Society, Michael Buckland Information and the Modern Corporation, James W. Cortada Intellectual Property Strategy, John Palfrey The Internet of Things, Samuel Greengard Machine Learning: The New AI, Ethem Alpaydın Memes in Digital Culture, Limor Shifman Metadata, Jeffrey Pomerantz The Mind–Body Problem, Jonathan Westphal MOOCs, Jonathan Haber Neuroplasticity, Moheb Costandi Open Access, Peter Suber Paradox, Margaret Cuonzo Robots, John Jordan Self-Tracking, Gina Neff and Dawn Nafus Sustainability, Kent E. Portney The Technological Singularity, Murray Shanahan Understanding Beliefs, Nils J. Nilsson Waves, Frederic Raichlen MACHINE LEARNING THE NEW AI ETHEM ALPAYDIN The MIT Press Cambridge, Massachusetts London, England © 2016 Massachusetts Institute of Technology All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from the publisher. This book was set in Chaparral and DIN by Toppan Best-set Premedia Limited. Printed and bound in the United States of America. Library of Congress Cataloging-in-Publication Data Names: Alpaydın, Ethem, author. Title: Machine learning : the new AI / Ethem Alpaydın. Description: Cambridge, MA : MIT Press, [2016] | Series: MIT Press essential knowledge series | Includes bibliographical references and index. Identifiers: LCCN 2016012342 | ISBN 9780262529518 (pbk. : alk. paper) Subjects: LCSH: Machine learning. | Artificial intelligence. Classification: LCC Q325.5 .A47 2016 | DDC 006.3/1—dc23 LC record available at https://lccn.loc.gov/2016012342 10 9 8 7 6 5 4 3 2 1 CONTENTS Series Foreword vii Preface ix 1 Why We Are Interested in Machine Learning 1 2 Machine Learning, Statistics, and Data Analytics 29 3 Pattern Recognition 55 4 Neural Networks and Deep Learning 85 5 Learning Clusters and Recommendations 111 6 Learning to Take Actions 125 7 Where Do We Go from Here? 141 Notes 169 Glossary 171 References 183 Further Readings 187 Index 189 SERIES FOREWORD The MIT Press Essential Knowledge series offers acces- sible, concise, beautifully produced pocket-size books on topics of current interest. Written by leading thinkers, the books in this series deliver expert overviews of subjects that range from the cultural and the historical to the scientific and the technical. In today’s era of instant information gratification, we have ready access to opinions, rationalizations, and super- ficial descriptions. Much harder to come by is the founda- tional knowledge that informs a principled understanding of the world. Essential Knowledge books fill that need. Synthesizing specialized subject matter for nonspecialists and engaging critical topics through fundamentals, each of these compact volumes offers readers a point of access to complex ideas. Bruce Tidor Professor of Biological Engineering and Computer Science Massachusetts Institute of Technology PREFACE A quiet revolution has been taking place in computer science for the last two decades. Nowadays, more and more, we see computer programs that learn—that is, software that can adapt their behavior automatically to better match the requirements of their task. We now have programs that learn to recognize people from their faces, un- derstand speech, drive a car, or recommend which movie to watch—with promises to do more in the future. Once, it used to be the programmer who defined what the computer had to do, by coding an algorithm in a programming language. Now for some tasks, we do not write programs but collect data. The data contains instances of what is to be done, and the learning algorithm modifies a learner program automatically in such a way so as to match the requirements specified in the data. Since the advent of computers in the middle of the last century, our lives have become increasingly computerized and digital. Computers are no longer just the numeric cal- culators they once were. Databases and digital media have taken the place of printing on paper as the main medium of information storage, and digital communication over computer networks supplanted the post as the main mode of information transfer. First with the personal computer with its easy-to-use graphical interface, and then with the x Preface phone and other smart devices, the computer has become a ubiquitous device, a household appliance just like the TV or the microwave. Nowadays, all sorts of information, not only numbers and text but also image, video, audio, and so on, are stored, processed, and—thanks to online con- nectivity—transferred digitally. All this digital processing results in a lot of data, and it is this surge of data—what we can call a “dataquake”—that is mainly responsible for triggering the widespread interest in data analysis and machine learning. For many applications—from vision to speech, from translation to robotics—we were not able to devise very good algorithms despite decades of research beginning in the 1950s. But for all these tasks it is easy to collect data, and now the idea is to learn the algorithms for these automatically from data, replacing programmers with learning programs. This is the niche of machine learning, and it is not only that the data continuously has got bigger in these last two decades, but also that the theory of machine learning to process that data to turn it into knowledge has advanced significantly. Today, in different types of business, from retail and finance to manufacturing, as our systems are computerized, more data is continuously generated and collected. This is also true in various fields of science, from astron- omy to biology. In our everyday lives too, as digital technology increasingly infiltrates our daily existence, as our Preface xi digital footprint deepens, not only as consumers and users but also through social media, an increasingly larger part of our lives is recorded and becomes data. Whatever its source—business, scientific, or personal—data that just lies dormant passively is not of any use, and smart people have been finding new ways to make use of that data and turn it into a useful product or service. In this transforma- tion, machine learning is playing a more significant role. Our belief is that behind all this seemingly complex and voluminous data, there lies a simple explanation. That although the data is big, it can be explained in terms of a relatively simple model with a small number of hidden factors and their interaction. Think about millions of cus- tomers who buy thousands of products online or from their local supermarket every day. This implies a very large database of transactions; but what saves us and works to our advantage is that there is a pattern to this data. People do not shop at random. A person throwing a party buys a certain subset of products, and a person who has a baby at home buys a different subset—there are hidden factors that explain customer behavior. It is this inference of a hidden model—namely, the underlying factors and their interaction—from the observed data that is at the core of machine learning. Machine learning is not just the commercial applica- tion of methods to extract information from data; learning is also a requisite of intelligence. An intelligent system xii Preface should be able to adapt to its environment; it should learn not to repeat its mistakes but to repeat its successes. Previ- ously, researchers used to believe that for artificial intelligence to become reality, we needed a new paradigm, a new type of thinking, a new model of computation, or a whole new set of algorithms. Taking into account the recent successes in machine learning in various domains, it can now be claimed that what we need is not a set of new specific algorithms but a lot of example data and sufficient computing power to run the learning methods on that much data, bootstrapping the necessary algorithms from data. It may be conjectured that tasks such as machine translation and planning can be solved with such learning algorithms that are relatively simple but trained on large amounts of example data—recent successes with “deep learning” support this claim. Intelligence seems not to originate from some outlandish formula, but rather from the patient, almost brute force use of simple, straightfor- ward algorithms. It seems that as technology develops and we get faster computers and more data, learning algorithms will gener- ate a slightly higher level of intelligence, which will find use in a new set of slightly smarter devices and software. It will not be surprising if this type of learned intelligence reaches the level of human intelligence some time before this century is over. Preface xiii While I was working on this book, one of the most prestigious scientific journals, Science, in its July 15, 2015, issue (vol. 349, no. 6245), published a special section on Artificial Intelligence. Though the title announces a focus on artificial intelligence, the dominant theme is machine learning. This is just another indicator that machine learning is now the driving force in artificial intelligence; after the disappointment of logic-based, programmed expert systems in 1980s, it has revived the field, delivering significant results. The aim of this book is to give the reader an overall idea about what machine learning is, the basics of some important learning algorithms, and a set of example applications. The book is intended for a general readership, and only the essentials of the learning methods are discussed without any mathematical or programming details. The book does not cover any of the machine-learning applications in much detail either; a number of examples are discussed just enough to give the fundamentals without going into the particulars. For more information on the machine learning algorithms, the reader can refer to my textbook on the topic, on which this book is heavily based: Ethem Alpaydın, Intro- duction to Machine Learning, 3rd ed. (Cambridge, MA: MIT Press, 2014).