Practical Machine Learning with Python: A Problem-Solver’s Guide to Building Real-World Intelligent

545 Pages·2017·13.27 MB·English

Practical Machine
Learning with
Python
A Problem-Solver’s Guide to Building
Real-World Intelligent Systems
—
Dipanjan Sarkar
Raghav Bali
Tushar Sharma
Practical Machine
Learning with Python
A Problem-Solver’s Guide to Building
Real-World Intelligent Systems
Dipanjan Sarkar
Raghav Bali
Tushar Sharma
Practical Machine Learning with Python
Dipanjan Sarkar Raghav Bali
Bangalore, Karnataka, India Bangalore, Karnataka, India
Tushar Sharma
Bangalore, Karnataka, India
ISBN-13 (pbk): 978-1-4842-3206-4 ISBN-13 (electronic): 978-1-4842-3207-1
https://doi.org/10.1007/978-1-4842-3207-1
Library of Congress Control Number: 2017963290
Copyright © 2018 by Dipanjan Sarkar, Raghav Bali and Tushar Sharma
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the
material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage
and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or
hereafter developed.
Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with
every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an
editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark.
The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are
not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to
proprietary rights.
While the advice and information in this book are believed to be true and accurate at the date of publication,
neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or
omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material
contained herein.
Cover image by Freepik (www.freepik.com)
Managing Director: Welmoed Spahr
Editorial Director: Todd Green
Acquisitions Editor: Celestin Suresh John
Development Editor: Matthew Moodie
Technical Reviewer: Jojo Moolayil
Coordinating Editor: Sanchita Mandal
Copy Editor: Kezia Endsley
Distributed to the book trade worldwide by Springer Science+Business Media New York,
233 Spring Street, 6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax [phone], e-mail
Tis book is dedicated to my parents, partner, friends, family, and well-wishers.
—Dipanjan Sarkar
To all my inspirations, who would never read this!
—Raghav Bali
Dedicated to my family and friends.
—Tushar Sharma
Contents
About the Authors ..................................................................................................xvii
About the Technical Reviewer ................................................................................xix
Acknowledgments ..................................................................................................xxi
Foreword ..............................................................................................................xxiii
Introduction ...........................................................................................................xxv
■ Part I: Understanding Machine Learning ............................................ 1
■ Chapter 1: Machine Learning Basics ..................................................................... 3
The Need for Machine Learning ....................................................................................... 4
Making Data-Driven Decisions ............................................................................................................... 4
Efficiency and Scale ............................................................................................................................... 5
T raditional Programming Paradigm ........................................................................................................ 5
Why Machine Learning? ......................................................................................................................... 6
Understanding Machine Learning .................................................................................... 8
Why Make Machines Learn?................................................................................................................... 8
Formal Definition .................................................................................................................................... 9
A Multi-Disciplinary Field ..................................................................................................................... 13
Computer Science .......................................................................................................... 14
Theoretical Computer Science.............................................................................................................. 15
P ractical Computer Science ................................................................................................................. 15
Important Concepts .............................................................................................................................. 15
D ata Science .................................................................................................................. 16
v
■ CONTENTS
Mathematics ...................................................................................................1..8.............
Important Concep t.s.........................................................................................................1..9..................
Statistics ........................................................................................................2..4..............
Data Mining .....................................................................................................2..5.............
Artificial Intelligenc .e........................................................................................2..5...........
Natural Language Processi n..g...........................................................................2..6.........
D eep Learning .................................................................................................2..8.............
Important Concep t.s.........................................................................................................3..1..................
Machine Learning Method ..s..............................................................................3..4..........
S upervised Learnin g.........................................................................................3..5...........
Classificatio n..................................................................................................................3..6....................
Regression ......................................................................................................................3..7....................
U nsupervised Learnin .g....................................................................................3..8...........
C lustering .......................................................................................................................3.9.....................
Dimensionality Reducti o..n.................................................................................................4.0.................
Anomaly Detection................................................................................................................................ 41
Association Rule-Mini n..g..................................................................................................4..1.................
Semi-Supervised Learnin ..g...............................................................................4..2..........
R einforcement Learnin .g...................................................................................4..2...........
Batch Learnin g................................................................................................4..3.............
Online Learnin g................................................................................................4.4.............
Instance Based Learnin ..g..................................................................................4.4...........
Model Based Learnin .g......................................................................................4..5...........
The CRISP-DM Process Mod .e..l..........................................................................4..5.........
Business Understandin ..g..................................................................................................4..6.................
Data Understandin .g.........................................................................................................4..8..................
Data Preparatio .n.............................................................................................................5..0...................
Modeling ........................................................................................................................5..1.....................
Evaluation .......................................................................................................................5.2.....................
Deployment........................................................................................................................................... 52
vi
■ CONTENTS
Building Machine Intelligenc .e................................................................................5..2.....
Machine Learning Pipeline .s....................................................................................................5..2..........
Supervised Machine Learning Pipeli n..e....................................................................................5..4........
Unsupervised Machine Learning Pipeli n..e.................................................................................5..5.......
Real-World Case Study: Predicting Student Grant Recommendati .o.n..s......................5.5
Objective ................................................................................................................................5..6.............
Data Retrieva l.........................................................................................................................5..6............
Data Preparation .....................................................................................................................5..7............
Modeling ................................................................................................................................6..0.............
Model Evaluation ....................................................................................................................6..1............
Model Deploymen .t..................................................................................................................6..1...........
Prediction in Actio .n.................................................................................................................6..2...........
Challenges in Machine Learnin .g............................................................................6..4.....
Real-World Applications of Machine Learni n..g........................................................6..4...
Summary ..............................................................................................................6..5........
■ Chapter 2: The Python Machine Learning Ecosyste .m.....................................6..7.
Python: An Introductio n..........................................................................................6..7......
Strengths ...............................................................................................................................6..8.............
Pitfalls ....................................................................................................................................6.8..............
Setting Up a Python Environme n..t............................................................................................6..9.........
Why Python for Data Science .?.................................................................................................7..1.........
Introducing the Python Machine Learning Ecosyste ..m.............................................7..2..
J upyter Notebook .s..................................................................................................................7..2...........
NumPy ...................................................................................................................................7..5.............
Pandas ...................................................................................................................................8..4.............
Scikit-learn ............................................................................................................................9..6.............
Neural Networks and Deep Learnin ..g......................................................................................1..0..2......
Text Analytics and Natural Language Process i.n..g....................................................................1..1..2....
Statsmodels ..........................................................................................................................1..1..6..........
Summary ............................................................................................................1..1..8......
vii
■ CONTENTS
■ Part II: The Machine Learning Pipelin .e........................................1..19
■ Chapter 3: Processing, Wrangling, and Visualizing Da ..t.a..............................1..21
D ata Collection ....................................................................................................1..2..2.....
C SV .....................................................................................................................................1..2..2............
J SON ....................................................................................................................................1.2..4............
XML .....................................................................................................................................1..2..8............
HTML and Scrapin g...............................................................................................................1..3..1.........
S QL .....................................................................................................................................1..3..6............
D ata Description ..................................................................................................1..3..7.....
Numeric ...............................................................................................................................1..3..7...........
Text ..................................................................................................................................................... 137
Categorical ......................................................................................................................................... 137
D ata Wrangling ............................................................................................................. 138
U nderstanding Data ............................................................................................................................ 138
F iltering Data ...................................................................................................................................... 141
Typecasting ......................................................................................................................................... 144
Transformations .................................................................................................................................. 144
Imputing Missing Values ..................................................................................................................... 145
H andling Duplicates ............................................................................................................................ 147
Handling Categorical Data .................................................................................................................. 147
N ormalizing Values ............................................................................................................................. 148
String Manipulations .......................................................................................................................... 149
Data Summarization ..................................................................................................... 149
Data Visualization ......................................................................................................... 151
Visualizing with Pandas ...................................................................................................................... 152
Visualizing with Matplotlib.................................................................................................................. 161
P ython Visualization Ecosystem ......................................................................................................... 176
Summary ...................................................................................................................... 176
viii
■ CONTENTS
■ Chapter 4: Feature Engineering and Select .i.o.n..........................................1..7..7.
Features: Understand Your Data Bet .t.e..r...........................................................1..7..8.....
Data and Dataset .s..........................................................................................................1..7..8...............
Features .........................................................................................................................1..7.9..................
Models ..........................................................................................................................1..7..9..................
Revisiting the Machine Learning Pipel .in..e.........................................................1..7..9....
Feature Extraction and Engineeri .n..g.................................................................1..8..1.....
What Is Feature Engineering?............................................................................................................. 181
Why Feature Engineerin g..?...............................................................................................1..8..3.............
How Do You Engineer Feature .s..?.......................................................................................1.8..4............
Feature Engineering on Numeric Da .t..a.............................................................1..8..5.....
Raw Measure s................................................................................................................1..8..5................
Binarizatio n.....................................................................................................................1.8..7.................
Rounding .......................................................................................................................1..8..8.................
I nteraction s.....................................................................................................................1.8..9.................
B inning ..........................................................................................................................1.9..1..................
Statistical Transformatio .n..s.............................................................................................1..9..7.............
Feature Engineering on Categorical D .a.t..a.........................................................2..0..0....
Transforming Nominal Featur .e..s.......................................................................................2.0..1............
Transforming Ordinal Featur .e..s........................................................................................2..0..2............
Encoding Categorical Featur .e..s........................................................................................2..0..3............
Feature Engineering on Text Da .t.a....................................................................2..0..9......
Text Pre-Processin .g.........................................................................................................2.1..0...............
Bag of Words Mod e..l........................................................................................................2.1..1...............
Bag of N-Grams Mod e..l...................................................................................................2..1..2..............
TF-IDF Mode .l.................................................................................................................2..1..3................
Document Similari t.y........................................................................................................2..1.4...............
T opic Model s...................................................................................................................2..1..6................
Word Embedding .s...........................................................................................................2..1..7...............
ix
■ CONTENTS
Feature Engineering on Temporal Data ........................................................................ 220
Date-Based Features .......................................................................................................................... 221
T ime-Based Features ......................................................................................................................... 222
F eature Engineering on Image Data ............................................................................. 224
I mage Metadata Features ................................................................................................................... 225
Raw Image and Channel Pixels .......................................................................................................... 225
Grayscale Image Pixels ....................................................................................................................... 227
B inning Image Intensity Distribution .................................................................................................. 227
Image Aggregation Statistics .............................................................................................................. 228
E dge Detection ................................................................................................................................... 229
Object Detection ................................................................................................................................. 230
Localized Feature Extraction .............................................................................................................. 231
Visual Bag of Words Model ................................................................................................................. 233
Automated Feature Engineering with Deep Learning ......................................................................... 236
Feature Scaling ............................................................................................................ 239
Standardized Scaling .......................................................................................................................... 240
M in-Max Scaling ................................................................................................................................. 240
R obust Scaling .................................................................................................................................... 241
Feature Selection ......................................................................................................... 242
Threshold-Based Methods .................................................................................................................. 243
S tatistical Methods ............................................................................................................................. 244
Recursive Feature Elimination ............................................................................................................ 247
M odel-Based Selection ....................................................................................................................... 248
Dimensionality Reduction ............................................................................................. 249
F eature Extraction with Principal Component Analysis ...................................................................... 250
Summary ...................................................................................................................... 252
■ Chapter 5: Building, Tuning, and Deploying Models .......................................... 255
B uilding Models ............................................................................................................ 256
M odel Types ........................................................................................................................................ 257
L earning a Model ................................................................................................................................ 260
M odel Building Examples ................................................................................................................... 263
x