Practical Machine Learning with Python: A Problem-Solver’s Guide to Building Real-World Intelligent PDF
Preview Practical Machine Learning with Python: A Problem-Solver’s Guide to Building Real-World Intelligent
Practical Machine Learning with Python A Problem-Solver’s Guide to Building Real-World Intelligent Systems — Dipanjan Sarkar Raghav Bali Tushar Sharma Practical Machine Learning with Python A Problem-Solver’s Guide to Building Real-World Intelligent Systems Dipanjan Sarkar Raghav Bali Tushar Sharma Practical Machine Learning with Python Dipanjan Sarkar Raghav Bali Bangalore, Karnataka, India Bangalore, Karnataka, India Tushar Sharma Bangalore, Karnataka, India ISBN-13 (pbk): 978-1-4842-3206-4 ISBN-13 (electronic): 978-1-4842-3207-1 https://doi.org/10.1007/978-1-4842-3207-1 Library of Congress Control Number: 2017963290 Copyright © 2018 by Dipanjan Sarkar, Raghav Bali and Tushar Sharma This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein. Cover image by Freepik (www.freepik.com) Managing Director: Welmoed Spahr Editorial Director: Todd Green Acquisitions Editor: Celestin Suresh John Development Editor: Matthew Moodie Technical Reviewer: Jojo Moolayil Coordinating Editor: Sanchita Mandal Copy Editor: Kezia Endsley Distributed to the book trade worldwide by Springer Science+Business Media New York, 233 Spring Street, 6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail Tis book is dedicated to my parents, partner, friends, family, and well-wishers. —Dipanjan Sarkar To all my inspirations, who would never read this! —Raghav Bali Dedicated to my family and friends. —Tushar Sharma Contents About the Authors ..................................................................................................xvii About the Technical Reviewer ................................................................................xix Acknowledgments ..................................................................................................xxi Foreword ..............................................................................................................xxiii Introduction ...........................................................................................................xxv ■ Part I: Understanding Machine Learning ............................................ 1 ■ Chapter 1: Machine Learning Basics ..................................................................... 3 The Need for Machine Learning ....................................................................................... 4 Making Data-Driven Decisions ............................................................................................................... 4 Efficiency and Scale ............................................................................................................................... 5 T raditional Programming Paradigm ........................................................................................................ 5 Why Machine Learning? ......................................................................................................................... 6 Understanding Machine Learning .................................................................................... 8 Why Make Machines Learn?................................................................................................................... 8 Formal Definition .................................................................................................................................... 9 A Multi-Disciplinary Field ..................................................................................................................... 13 Computer Science .......................................................................................................... 14 Theoretical Computer Science.............................................................................................................. 15 P ractical Computer Science ................................................................................................................. 15 Important Concepts .............................................................................................................................. 15 D ata Science .................................................................................................................. 16 v ■ CONTENTS Mathematics ...................................................................................................1..8............. Important Concep t.s.........................................................................................................1..9.................. Statistics ........................................................................................................2..4.............. Data Mining .....................................................................................................2..5............. Artificial Intelligenc .e........................................................................................2..5........... Natural Language Processi n..g...........................................................................2..6......... D eep Learning .................................................................................................2..8............. Important Concep t.s.........................................................................................................3..1.................. Machine Learning Method ..s..............................................................................3..4.......... S upervised Learnin g.........................................................................................3..5........... Classificatio n..................................................................................................................3..6.................... Regression ......................................................................................................................3..7.................... U nsupervised Learnin .g....................................................................................3..8........... C lustering .......................................................................................................................3.9..................... Dimensionality Reducti o..n.................................................................................................4.0................. Anomaly Detection................................................................................................................................ 41 Association Rule-Mini n..g..................................................................................................4..1................. Semi-Supervised Learnin ..g...............................................................................4..2.......... R einforcement Learnin .g...................................................................................4..2........... Batch Learnin g................................................................................................4..3............. Online Learnin g................................................................................................4.4............. Instance Based Learnin ..g..................................................................................4.4........... Model Based Learnin .g......................................................................................4..5........... The CRISP-DM Process Mod .e..l..........................................................................4..5......... Business Understandin ..g..................................................................................................4..6................. Data Understandin .g.........................................................................................................4..8.................. Data Preparatio .n.............................................................................................................5..0................... Modeling ........................................................................................................................5..1..................... Evaluation .......................................................................................................................5.2..................... Deployment........................................................................................................................................... 52 vi ■ CONTENTS Building Machine Intelligenc .e................................................................................5..2..... Machine Learning Pipeline .s....................................................................................................5..2.......... Supervised Machine Learning Pipeli n..e....................................................................................5..4........ Unsupervised Machine Learning Pipeli n..e.................................................................................5..5....... Real-World Case Study: Predicting Student Grant Recommendati .o.n..s......................5.5 Objective ................................................................................................................................5..6............. Data Retrieva l.........................................................................................................................5..6............ Data Preparation .....................................................................................................................5..7............ Modeling ................................................................................................................................6..0............. Model Evaluation ....................................................................................................................6..1............ Model Deploymen .t..................................................................................................................6..1........... Prediction in Actio .n.................................................................................................................6..2........... Challenges in Machine Learnin .g............................................................................6..4..... Real-World Applications of Machine Learni n..g........................................................6..4... Summary ..............................................................................................................6..5........ ■ Chapter 2: The Python Machine Learning Ecosyste .m.....................................6..7. Python: An Introductio n..........................................................................................6..7...... Strengths ...............................................................................................................................6..8............. Pitfalls ....................................................................................................................................6.8.............. Setting Up a Python Environme n..t............................................................................................6..9......... Why Python for Data Science .?.................................................................................................7..1......... Introducing the Python Machine Learning Ecosyste ..m.............................................7..2.. J upyter Notebook .s..................................................................................................................7..2........... NumPy ...................................................................................................................................7..5............. Pandas ...................................................................................................................................8..4............. Scikit-learn ............................................................................................................................9..6............. Neural Networks and Deep Learnin ..g......................................................................................1..0..2...... Text Analytics and Natural Language Process i.n..g....................................................................1..1..2.... Statsmodels ..........................................................................................................................1..1..6.......... Summary ............................................................................................................1..1..8...... vii ■ CONTENTS ■ Part II: The Machine Learning Pipelin .e........................................1..19 ■ Chapter 3: Processing, Wrangling, and Visualizing Da ..t.a..............................1..21 D ata Collection ....................................................................................................1..2..2..... C SV .....................................................................................................................................1..2..2............ J SON ....................................................................................................................................1.2..4............ XML .....................................................................................................................................1..2..8............ HTML and Scrapin g...............................................................................................................1..3..1......... S QL .....................................................................................................................................1..3..6............ D ata Description ..................................................................................................1..3..7..... Numeric ...............................................................................................................................1..3..7........... Text ..................................................................................................................................................... 137 Categorical ......................................................................................................................................... 137 D ata Wrangling ............................................................................................................. 138 U nderstanding Data ............................................................................................................................ 138 F iltering Data ...................................................................................................................................... 141 Typecasting ......................................................................................................................................... 144 Transformations .................................................................................................................................. 144 Imputing Missing Values ..................................................................................................................... 145 H andling Duplicates ............................................................................................................................ 147 Handling Categorical Data .................................................................................................................. 147 N ormalizing Values ............................................................................................................................. 148 String Manipulations .......................................................................................................................... 149 Data Summarization ..................................................................................................... 149 Data Visualization ......................................................................................................... 151 Visualizing with Pandas ...................................................................................................................... 152 Visualizing with Matplotlib.................................................................................................................. 161 P ython Visualization Ecosystem ......................................................................................................... 176 Summary ...................................................................................................................... 176 viii ■ CONTENTS ■ Chapter 4: Feature Engineering and Select .i.o.n..........................................1..7..7. Features: Understand Your Data Bet .t.e..r...........................................................1..7..8..... Data and Dataset .s..........................................................................................................1..7..8............... Features .........................................................................................................................1..7.9.................. Models ..........................................................................................................................1..7..9.................. Revisiting the Machine Learning Pipel .in..e.........................................................1..7..9.... Feature Extraction and Engineeri .n..g.................................................................1..8..1..... What Is Feature Engineering?............................................................................................................. 181 Why Feature Engineerin g..?...............................................................................................1..8..3............. How Do You Engineer Feature .s..?.......................................................................................1.8..4............ Feature Engineering on Numeric Da .t..a.............................................................1..8..5..... Raw Measure s................................................................................................................1..8..5................ Binarizatio n.....................................................................................................................1.8..7................. Rounding .......................................................................................................................1..8..8................. I nteraction s.....................................................................................................................1.8..9................. B inning ..........................................................................................................................1.9..1.................. Statistical Transformatio .n..s.............................................................................................1..9..7............. Feature Engineering on Categorical D .a.t..a.........................................................2..0..0.... Transforming Nominal Featur .e..s.......................................................................................2.0..1............ Transforming Ordinal Featur .e..s........................................................................................2..0..2............ Encoding Categorical Featur .e..s........................................................................................2..0..3............ Feature Engineering on Text Da .t.a....................................................................2..0..9...... Text Pre-Processin .g.........................................................................................................2.1..0............... Bag of Words Mod e..l........................................................................................................2.1..1............... Bag of N-Grams Mod e..l...................................................................................................2..1..2.............. TF-IDF Mode .l.................................................................................................................2..1..3................ Document Similari t.y........................................................................................................2..1.4............... T opic Model s...................................................................................................................2..1..6................ Word Embedding .s...........................................................................................................2..1..7............... ix ■ CONTENTS Feature Engineering on Temporal Data ........................................................................ 220 Date-Based Features .......................................................................................................................... 221 T ime-Based Features ......................................................................................................................... 222 F eature Engineering on Image Data ............................................................................. 224 I mage Metadata Features ................................................................................................................... 225 Raw Image and Channel Pixels .......................................................................................................... 225 Grayscale Image Pixels ....................................................................................................................... 227 B inning Image Intensity Distribution .................................................................................................. 227 Image Aggregation Statistics .............................................................................................................. 228 E dge Detection ................................................................................................................................... 229 Object Detection ................................................................................................................................. 230 Localized Feature Extraction .............................................................................................................. 231 Visual Bag of Words Model ................................................................................................................. 233 Automated Feature Engineering with Deep Learning ......................................................................... 236 Feature Scaling ............................................................................................................ 239 Standardized Scaling .......................................................................................................................... 240 M in-Max Scaling ................................................................................................................................. 240 R obust Scaling .................................................................................................................................... 241 Feature Selection ......................................................................................................... 242 Threshold-Based Methods .................................................................................................................. 243 S tatistical Methods ............................................................................................................................. 244 Recursive Feature Elimination ............................................................................................................ 247 M odel-Based Selection ....................................................................................................................... 248 Dimensionality Reduction ............................................................................................. 249 F eature Extraction with Principal Component Analysis ...................................................................... 250 Summary ...................................................................................................................... 252 ■ Chapter 5: Building, Tuning, and Deploying Models .......................................... 255 B uilding Models ............................................................................................................ 256 M odel Types ........................................................................................................................................ 257 L earning a Model ................................................................................................................................ 260 M odel Building Examples ................................................................................................................... 263 x
The list of books you might like
