Machine Learning Using R

580 Pages·2017·11.47 MB·English

by  Karthik Ramasubramanian

Machine
Learning Using R
A Comprehensive Guide to Machine
Learning
—
Karthik Ramasubramanian
Abhishek Singh
Machine Learning
Using R
Karthik Ramasubramanian
Abhishek Singh
Machine Learning Using R
Karthik Ramasubramanian Abhishek Singh
New Delhi, Delhi, India New Delhi, Delhi, India
ISBN-13 (pbk): 978-1-4842-2333-8 ISBN-13 (electronic): 978-1-4842-2334-5
DOI 10.1007/978-1-4842-2334-5
Library of Congress Control Number: 2016961515
Copyright © 2017 Karthik Ramasubramanian and Abhishek Singh
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole
or part of the material is concerned, specifically the rights of translation, reprinting, reuse of
illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical
way, and transmission or information storage and retrieval, electronic adaptation, computer
software, or by similar or dissimilar methodology now known or hereafter developed.
Trademarked names, logos, and images may appear in this book. Rather than use a trademark
symbol with every occurrence of a trademarked name, logo, or image we use the names, logos,
and images only in an editorial fashion and to the benefit of the trademark owner, with no
intention of infringement of the trademark.
The use in this publication of trade names, trademarks, service marks, and similar terms, even
if they are not identified as such, is not to be taken as an expression of opinion as to whether or
not they are subject to proprietary rights.
While the advice and information in this book are believed to be true and accurate at the
date of publication, neither the authors nor the editors nor the publisher can accept any legal
responsibility for any errors or omissions that may be made. The publisher makes no warranty,
express or implied, with respect to the material contained herein.
Managing Director: Welmoed Spahr
Acquisitions Editor: Celestin Suresh John
Development Editor: James Markham
Technical Reviewer: Jojo Moolayil
Editorial Board: Steve Anglin, Pramila Balen, Laura Berendson, Aaron Black,
Louise Corrigan, Jonathan Gennick, Robert Hutchinson, Celestin Suresh John,
Nikhil Karkal, James Markham, Susan McDermott, Matthew Moodie, Natalie Pao,
Gwenan Spearing
Coordinating Editor: Sanchita Mandal
Copy Editor: Lori Jacobs
Compositor: SPi Global
Indexer: SPi Global
Cover Image: Freepik
Distributed to the book trade worldwide by Springer Science+Business Media New York,
233 Spring Street, 6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax [phone],
e-mail [email protected], or visit www.springer.com. Apress Media, LLC is a
California LLC and the sole member (owner) is Springer Science + Business Media Finance Inc
(SSBM Finance Inc). SSBM Finance Inc is a Delaware corporation.
For information on translations, please e-mail [email protected], or visit www.apress.com.
Apress and friends of ED books may be purchased in bulk for academic, corporate, or promotional
use. eBook versions and licenses are also available for most titles. For more information, reference
our Special Bulk Sales–eBook Licensing web page at www.apress.com/bulk-sales.
Any source code or other supplementary materials referenced by the author in this text is available
to readers at www.apress.com. For detailed information about how to locate your book’s source
code, go to www.apress.com/source-code/.
Printed on acid-free paper
To our parents for being the guiding light and a strong
pillar of support.
And to our decade-long friendship.
Contents at a Glance
About the Authors ���������������������������������������������������������������������������xix
About the Technical Reviewer ��������������������������������������������������������xxi
Acknowledgments ������������������������������������������������������������������������xxiii
■ Chapter 1: Introduction to Machine Learning and R �����������������������1
■ Chapter 2: Data Preparation and Exploration �������������������������������31
■ Chapter 3: Sampling and Resampling Techniques �����������������������67
■ Chapter 4: Data Visualization in R ����������������������������������������������129
■ Chapter 5: Feature Engineering ��������������������������������������������������181
■ Chapter 6: Machine Learning Theory and Practices �������������������219
■ Chapter 7: Machine Learning Model Evaluation �������������������������425
■ Chapter 8: Model Performance Improvement �����������������������������465
■ Chapter 9: Scalable Machine Learning and Related
Technologies �������������������������������������������������������������������������������519
Index ����������������������������������������������������������������������������������������������555
v
Contents
About the Authors ���������������������������������������������������������������������������xix
About the Technical Reviewer ��������������������������������������������������������xxi
Acknowledgments ������������������������������������������������������������������������xxiii
■ Chapter 1: Introduction to Machine Learning and R �����������������������1
1.1 Understanding the Evolution ..........................................................2
1.1.1 Statistical Learning ......................................................................................2
1.1.2 Machine Learning (ML) .................................................................................3
1.1.3 Artificial Intelligence (AI)...............................................................................3
1.1.4 Data Mining ..................................................................................................4
1.1.5 Data Science ................................................................................................5
1.2 Probability and Statistics ...............................................................6
1.2.1 Counting and Probability Definition ..............................................................7
1.2.2 Events and Relationships .............................................................................9
1.2.3 Randomness, Probability, and Distributions ...............................................12
1.2.4 Confidence Interval and Hypothesis Testing ...............................................13
1.3 Getting Started with R ..................................................................18
1.3.1 Basic Building Blocks .................................................................................18
1.3.2 Data Structures in R ...................................................................................19
1.3.3 Subsetting ..................................................................................................21
1.3.4 Functions and Apply Family ........................................................................23
vii
■ Contents
1.4 Machine Learning Process Flow ..................................................26
1.4.1 Plan ............................................................................................................26
1.4.2 Explore........................................................................................................26
1.4.3 Build ...........................................................................................................27
1.4.4 Evaluate ......................................................................................................27
1.5 Other Technologies .......................................................................28
1.6 Summary ......................................................................................28
1.7 References ...................................................................................28
■ Chapter 2: Data Preparation and Exploration �������������������������������31
2.1 Planning the Gathering of Data ....................................................32
2.1.1 Variables Types ...........................................................................................32
2.1.2 Data Formats ..............................................................................................33
2.1.3 Data Sources ..............................................................................................40
2.2 Initial Data Analysis (IDA) .............................................................41
2.2.1 Discerning a First Look ...............................................................................41
2.2.2 Organizing Multiple Sources of Data into One ............................................43
2.2.3 Cleaning the Data .......................................................................................46
2.2.4 Supplementing with More Information .......................................................49
2.2.5 Reshaping ...................................................................................................50
2.3 Exploratory Data Analysis.............................................................51
2.3.1 Summary Statistics ....................................................................................52
2.3.2 Moment ......................................................................................................55
2.4 Case Study: Credit Card Fraud .....................................................61
2.4.1 Data Import .................................................................................................61
2.4.2 Data Transformation ...................................................................................62
2.4.3 Data Exploration .........................................................................................63
2.5 Summary ......................................................................................65
2.6 References ...................................................................................65
viii
■ Contents
■ Chapter 3: Sampling and Resampling Techniques �����������������������67
3.1 Introduction to Sampling ..............................................................68
3.2 Sampling Terminology ..................................................................69
3.2.1 Sample .......................................................................................................69
3.2.2 Sampling Distribution .................................................................................70
3.2.3 Population Mean and Variance ...................................................................70
3.2.4 Sample Mean and Variance ........................................................................70
3.2.5 Pooled Mean and Variance .........................................................................70
3.2.6 Sample Point ..............................................................................................71
3.2.7 Sampling Error ...........................................................................................71
3.2.8 Sampling Fraction ......................................................................................72
3.2.9 Sampling Bias ............................................................................................72
3.2.10 Sampling Without Replacement (SWOR) ..................................................72
3.2.11 Sampling with Replacement (SWR) ..........................................................72
3.3 Credit Card Fraud: Population Statistics.......................................73
3.3.1 Data Description .........................................................................................73
3.3.2 Population Mean .........................................................................................74
3.3.3 Population Variance ....................................................................................74
3.3.4 Pooled Mean and Variance .........................................................................75
3.4 Business Implications of Sampling ..............................................78
3.4.1 Features of Sampling .................................................................................79
3.4.2 Shortcomings of Sampling .........................................................................79
3.5 Probability and Non-Probability Sampling ...................................79
3.5.1 T ypes of Non-Probability Sampling.............................................................80
3.6 Statistical Theory on Sampling Distributions ...............................81
3.6.1 Law of Large Numbers: LLN ......................................................................81
3.6.2 Central Limit Theorem ................................................................................85
ix
■ Contents
3.7 Probability Sampling Techniques .................................................89
3.7.1 Population Statistics ...................................................................................89
3.7.2 Simple Random Sampling ..........................................................................93
3.7.3 Systematic Random Sampling .................................................................100
3.7.4 Stratified Random Sampling.....................................................................104
3.7.5 Cluster Sampling ......................................................................................111
3.7.6 Bootstrap Sampling ..................................................................................117
3.8 Monte Carlo Method: Acceptance-Rejection Method .................124
3.9 A Qualitative Account of Computational Savings by Sampling ...126
3.10 Summary ..................................................................................127
■ Chapter 4: Data Visualization in R ����������������������������������������������129
4.1 Introduction to the ggplot2 Package ..........................................130
4.2 World Development Indicators ...................................................132
4.3 Line Chart ...................................................................................132
4.4 Stacked Column Charts ..............................................................138
4.5 Scatterplots ...............................................................................144
4.6 Boxplots .....................................................................................145
4.7 Histograms and Density Plots ....................................................148
4.8 Pie Charts ...................................................................................152
4.9 Correlation Plots .........................................................................154
4.10 HeatMaps .................................................................................156
4.11 Bubble Charts ...........................................................................158
4.12 Waterfall Charts ........................................................................162
4.13 Dendogram ...............................................................................165
4.14 Wordclouds...............................................................................167
4.15 Sankey Plots ............................................................................169
4.16 T ime Series Graphs ..................................................................170
x
■ Contents
4.17 Cohort Diagrams ......................................................................172
4.18 Spatial Maps ............................................................................174
4.19 Summary ..................................................................................178
4.20 References ...............................................................................179
■ Chapter 5: Feature Engineering ��������������������������������������������������181
5.1 Introduction to Feature Engineering ...........................................182
5.1.1 Filter Methods ..........................................................................................184
5.1.2 Wrapper Methods .....................................................................................184
5.1.3 Embedded Methods ..................................................................................184
5.2 Understanding the Working Data ...............................................185
5.2.1 Data Summary ..........................................................................................186
5.2.2 Properties of Dependent Variable .............................................................186
5.2.3 Features Availability: Continuous or Categorical ......................................189
5.2.4 Setting Up Data Assumptions ...................................................................191
5.3 Feature Ranking .........................................................................191
5.4 Variable Subset Selection ..........................................................195
5.4.1 Filter Method ............................................................................................195
5.4.2 Wrapper Methods .....................................................................................199
5.4.3 Embedded Methods ..................................................................................206
5.5 Dimensionality Reduction ..........................................................210
5.6 Feature Engineering Checklist ...................................................215
5.7 Summary ....................................................................................217
5.8 References .................................................................................217
■ Chapter 6: Machine Learning Theory and Practices �������������������219
6.1 Machine Learning Types ............................................................222
6.1.1 Supervised Learning ................................................................................222
6.1.2 Unsupervised Learning .............................................................................223
xi