ebook img

Learning Tensorflow: A Guide to Building Deep Learning Systems PDF

242 Pages·2017·12.64 MB·English
Save to my drive
Quick download
Download
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview Learning Tensorflow: A Guide to Building Deep Learning Systems

Learning TensorFlow A GUIDE TO BUILDING DEEP LEARNING SYSTEMS Tom Hope, Yehezkel S. Resheff & Itay Lieder Learning TensorFlow A Guide to Building Deep Learning Systems Tom Hope, Yehezkel S. Resheff, and Itay Lieder BBeeiijjiinngg BBoossttoonn FFaarrnnhhaamm SSeebbaassttooppooll TTookkyyoo Learning TensorFlow by Tom Hope, Yehezkel S. Resheff, and Itay Lieder Copyright © 2017 Tom Hope, Itay Lieder, and Yehezkel S. Resheff. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com/safari). For more information, contact our corporate/insti‐ tutional sales department: 800-998-9938 or [email protected]. Editor: Nicole Tache Indexer: Judith McConville Production Editor: Shiny Kalapurakkel Interior Designer: David Futato Copyeditor: Rachel Head Cover Designer: Karen Montgomery Proofreader: Sharon Wilkey Illustrator: Rebecca Demarest August 2017: First Edition Revision History for the First Edition 2017-08-04: First Release 2017-09-15: Second Release The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Learning TensorFlow, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. While the publisher and the authors have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights. 978-1-491-97851-1 [LSI] Table of Contents Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii 1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Going Deep 1 Using TensorFlow for AI Systems 2 TensorFlow: What’s in a Name? 5 A High-Level Overview 6 Summary 8 2. Go with the Flow: Up and Running with TensorFlow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Installing TensorFlow 9 Hello World 11 MNIST 13 Softmax Regression 14 Summary 21 3. Understanding TensorFlow Basics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Computation Graphs 23 What Is a Computation Graph? 23 The Benefits of Graph Computations 24 Graphs, Sessions, and Fetches 24 Creating a Graph 25 Creating a Session and Running It 26 Constructing and Managing Our Graph 27 Fetches 29 Flowing Tensors 30 Nodes Are Operations, Edges Are Tensor Objects 30 Data Types 32 iii Tensor Arrays and Shapes 33 Names 37 Variables, Placeholders, and Simple Optimization 38 Variables 38 Placeholders 39 Optimization 40 Summary 49 4. Convolutional Neural Networks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 Introduction to CNNs 51 MNIST: Take II 53 Convolution 54 Pooling 56 Dropout 57 The Model 57 CIFAR10 61 Loading the CIFAR10 Dataset 62 Simple CIFAR10 Models 64 Summary 68 5. Text I: Working with Text and Sequences, and TensorBoard Visualization. . . . . . . . . . . 69 The Importance of Sequence Data 69 Introduction to Recurrent Neural Networks 70 Vanilla RNN Implementation 72 TensorFlow Built-in RNN Functions 82 RNN for Text Sequences 84 Text Sequences 84 Supervised Word Embeddings 88 LSTM and Using Sequence Length 89 Training Embeddings and the LSTM Classifier 91 Summary 93 6. Text II: Word Vectors, Advanced RNN, and Embedding Visualization. . . . . . . . . . . . . . . 95 Introduction to Word Embeddings 95 Word2vec 97 Skip-Grams 98 Embeddings in TensorFlow 100 The Noise-Contrastive Estimation (NCE) Loss Function 101 Learning Rate Decay 101 Training and Visualizing with TensorBoard 102 Checking Out Our Embeddings 103 Pretrained Embeddings, Advanced RNN 105 iv | Table of Contents Pretrained Word Embeddings 106 Bidirectional RNN and GRU Cells 110 Summary 112 7. TensorFlow Abstractions and Simplifications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 Chapter Overview 113 High-Level Survey 115 contrib.learn 117 Linear Regression 118 DNN Classifier 120 FeatureColumn 123 Homemade CNN with contrib.learn 128 TFLearn 131 Installation 131 CNN 131 RNN 134 Keras 136 Pretrained models with TF-Slim 143 Summary 151 8. Queues, Threads, and Reading Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 The Input Pipeline 153 TFRecords 154 Writing with TFRecordWriter 155 Queues 157 Enqueuing and Dequeuing 157 Multithreading 159 Coordinator and QueueRunner 160 A Full Multithreaded Input Pipeline 162 tf.train.string_input_producer() and tf.TFRecordReader() 164 tf.train.shuffle_batch() 164 tf.train.start_queue_runners() and Wrapping Up 165 Summary 166 9. Distributed TensorFlow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 Distributed Computing 167 Where Does the Parallelization Take Place? 168 What Is the Goal of Parallelization? 168 TensorFlow Elements 169 tf.app.flags 169 Clusters and Servers 170 Replicating a Computational Graph Across Devices 171 Table of Contents | v Managed Sessions 171 Device Placement 172 Distributed Example 173 Summary 179 10. Exporting and Serving Models with TensorFlow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 Saving and Exporting Our Model 181 Assigning Loaded Weights 182 The Saver Class 185 Introduction to TensorFlow Serving 191 Overview 192 Installation 193 Building and Exporting 194 Summary 201 A. Tips on Model Construction and Using TensorFlow Serving. . . . . . . . . . . . . . . . . . . . . . . 203 Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221 vi | Table of Contents Preface Deep learning has emerged in the last few years as a premier technology for building intelligent systems that learn from data. Deep neural networks, originally roughly inspired by how the human brain learns, are trained with large amounts of data to solve complex tasks with unprecedented accuracy. With open source frameworks making this technology widely available, it is becoming a must-know for anybody involved with big data and machine learning. TensorFlow is currently the leading open source software for deep learning, used by a rapidly growing number of practitioners working on computer vision, natural lan‐ guage processing (NLP), speech recognition, and general predictive analytics. This book is an end-to-end guide to TensorFlow designed for data scientists, engi‐ neers, students, and researchers. The book adopts a hands-on approach suitable for a broad technical audience, allowing beginners a gentle start while diving deep into advanced topics and showing how to build production-ready systems. In this book you will learn how to: 1. Get up and running with TensorFlow, rapidly and painlessly. 2. Use TensorFlow to build models from the ground up. 3. Train and understand popular deep learning models for computer vision and NLP. 4. Use extensive abstraction libraries to make development easier and faster. 5. Scale up TensorFlow with queuing and multithreading, training on clusters, and serving output in production. 6. And much more! This book is written by data scientists with extensive R&D experience in both indus‐ try and academic research. The authors take a hands-on approach, combining practi‐ cal and intuitive examples, illustrations, and insights suitable for practitioners seeking to build production-ready systems, as well as readers looking to learn to understand and build flexible and powerful models. vii Prerequisites This book assumes some basic Python programming know-how, including basic familiarity with the scientific library NumPy. Machine learning concepts are touched upon and intuitively explained throughout the book. For readers who want to gain a deeper understanding, a reasonable level of knowledge in machine learning, linear algebra, calculus, probability, and statistics is recommended. Conventions Used in This Book The following typographical conventions are used in this book: Italic Indicates new terms, URLs, email addresses, filenames, and file extensions. Constant width Used for program listings, as well as within paragraphs to refer to program ele‐ ments such as variable or function names, databases, data types, environment variables, statements, and keywords. Constant width bold Shows commands or other text that should be typed literally by the user. Constant width italic Shows text that should be replaced with user-supplied values or by values deter‐ mined by context. Using Code Examples Supplemental material (code examples, exercises, etc.) is available for download at https://github.com/Hezi-Resheff/Oreilly-Learning-TensorFlow. This book is here to help you get your job done. In general, if example code is offered with this book, you may use it in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing a CD-ROM of examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a signifi‐ cant amount of example code from this book into your product’s documentation does require permission. We appreciate, but do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Learning TensorFlow by Tom Hope, viii | Preface

See more

The list of books you might like

Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.