Statistical Regression and Classification: From Linear Models to Machine Learning

532 Pages·2017·4.61 MB·English

by  Norman Matloff

Statistical Regression
and Classification
From Linear Models to
Machine Learning
CHAPMAN & HALL/CRC
Texts in Statistical Science Series
Series Editors
Joseph K. Blitzstein, Harvard University, USA
Julian J. Faraway, University of Bath, UK
Martin Tanner, Northwestern University, USA
Jim Zidek, University of British Columbia, Canada
Statistical Theory: A Concise Introduction Problem Solving: A Statistician’s Guide,
F. Abramovich and Y. Ritov Second Edition
C. Chatfield
Practical Multivariate Analysis, Fifth Edition
A. Afifi, S. May, and V.A. Clark Statistics for Technology: A Course in Applied
Statistics, Third Edition
Practical Statistics for Medical Research
C. Chatfield
D.G. Altman
Analysis of Variance, Design, and Regression :
Interpreting Data: A First Course
Linear Modeling for Unbalanced Data,
in Statistics
Second Edition
A.J.B. Anderson
R. Christensen
Introduction to Probability with R
K. Baclawski Bayesian Ideas and Data Analysis: An
Introduction for Scientists and Statisticians
Linear Algebra and Matrix Analysis for
R. Christensen, W. Johnson, A. Branscum,
Statistics
and T.E. Hanson
S. Banerjee and A. Roy
Modelling Binary Data, Second Edition
Modern Data Science with R
D. Collett
B. S. Baumer, D. T. Kaplan, and N. J. Horton
Modelling Survival Data in Medical Research,
Mathematical Statistics: Basic Ideas and
Third Edition
Selected Topics, Volume I,
D. Collett
Second Edition
P. J. Bickel and K. A. Doksum Introduction to Statistical Methods for
Clinical Trials
Mathematical Statistics: Basic Ideas and
T.D. Cook and D.L. DeMets
Selected Topics, Volume II
P. J. Bickel and K. A. Doksum Applied Statistics: Principles and Examples
Analysis of Categorical Data with R D.R. Cox and E.J. Snell
C. R. Bilder and T. M. Loughin Multivariate Survival Analysis and Competing
Statistical Methods for SPC and TQM Risks
D. Bissell M. Crowder
Introduction to Probability Statistical Analysis of Reliability Data
J. K. Blitzstein and J. Hwang M.J. Crowder, A.C. Kimber,
T.J. Sweeting, and R.L. Smith
Bayesian Methods for Data Analysis,
Third Edition An Introduction to Generalized
B.P. Carlin and T.A. Louis Linear Models, Third Edition
A.J. Dobson and A.G. Barnett
Second Edition
R. Caulcutt Nonlinear Time Series: Theory, Methods, and
Applications with R Examples
The Analysis of Time Series: An Introduction,
R. Douc, E. Moulines, and D.S. Stoffer
Sixth Edition
C. Chatfield Introduction to Optimization Methods and
Their Applications in Statistics
Introduction to Multivariate Analysis
B.S. Everitt
C. Chatfield and A.J. Collins
Extending the Linear Model with R: Graphics for Statistics and Data Analysis with R
Generalized Linear, Mixed Effects and K.J. Keen
Nonparametric Regression Models, Second
Mathematical Statistics
Edition
K. Knight
J.J. Faraway
Introduction to Functional Data Analysis
Linear Models with R, Second Edition
P. Kokoszka and M. Reimherr
J.J. Faraway
Introduction to Multivariate Analysis:
A Course in Large Sample Theory
Linear and Nonlinear Modeling
T.S. Ferguson
S. Konishi
Multivariate Statistics: A Practical
Nonparametric Methods in Statistics with SAS
Approach
Applications
B. Flury and H. Riedwyl
O. Korosteleva
Readings in Decision Analysis
Modeling and Analysis of Stochastic Systems,
S. French
Third Edition
Discrete Data Analysis with R: Visualization V.G. Kulkarni
and Modeling Techniques for Categorical and
Exercises and Solutions in Biostatistical Theory
Count Data
L.L. Kupper, B.H. Neelon, and S.M. O’Brien
M. Friendly and D. Meyer
Exercises and Solutions in Statistical Theory
Markov Chain Monte Carlo:
L.L. Kupper, B.H. Neelon, and S.M. O’Brien
Stochastic Simulation for Bayesian Inference,
Design and Analysis of Experiments with R
Second Edition
J. Lawson
D. Gamerman and H.F. Lopes
Design and Analysis of Experiments with SAS
Bayesian Data Analysis, Third Edition
J. Lawson
A. Gelman, J.B. Carlin, H.S. Stern, D.B. Dunson,
A. Vehtari, and D.B. Rubin A Course in Categorical Data Analysis
T. Leonard
Multivariate Analysis of Variance and
Repeated Measures: A Practical Approach for Statistics for Accountants
Behavioural Scientists S. Letchford
D.J. Hand and C.C. Taylor
Introduction to the Theory of Statistical
Practical Longitudinal Data Analysis Inference
D.J. Hand and M. Crowder H. Liero and S. Zwanzig
Logistic Regression Models Statistical Theory, Fourth Edition
J.M. Hilbe B.W. Lindgren
Richly Parameterized Linear Models: Stationary Stochastic Processes: Theory and
Additive, Time Series, and Spatial Models Applications
Using Random Effects G. Lindgren
J.S. Hodges
Statistics for Finance
Statistics for Epidemiology E. Lindström, H. Madsen, and J. N. Nielsen
N.P. Jewell
The BUGS Book: A Practical Introduction to
Stochastic Processes: An Introduction, Bayesian Analysis
Second Edition D. Lunn, C. Jackson, N. Best, A. Thomas, and
P.W. Jones and P. Smith D. Spiegelhalter
The Theory of Linear Models Introduction to General and Generalized
B. Jørgensen Linear Models
H. Madsen and P. Thyregod
Pragmatics of Uncertainty
J.B. Kadane Time Series Analysis
H. Madsen
Principles of Uncertainty
J.B. Kadane Pólya Urn Models
H. Mahmoud
Randomization, Bootstrap and Monte Carlo Sampling Methodologies with Applications
Methods in Biology, Third Edition P.S.R.S. Rao
B.F.J. Manly
A First Course in Linear Model Theory
Statistical Regression and Classification: From N. Ravishanker and D.K. Dey
Linear Models to Machine Learning
Essential Statistics, Fourth Edition
N. Matloff
D.A.G. Rees
Introduction to Randomized Controlled
Stochastic Modeling and Mathematical
Clinical Trials, Second Edition
Statistics: A Text for Statisticians and
J.N.S. Matthews
Quantitative Scientists
Statistical Rethinking: A Bayesian Course with F.J. Samaniego
Examples in R and Stan
Statistical Methods for Spatial Data Analysis
R. McElreath
O. Schabenberger and C.A. Gotway
Statistical Methods in Agriculture and
Bayesian Networks: With Examples in R
Experimental Biology, Second Edition
M. Scutari and J.-B. Denis
R. Mead, R.N. Curnow, and A.M. Hasted
Large Sample Methods in Statistics
Statistics in Engineering: A Practical Approach
P.K. Sen and J. da Motta Singer
A.V. Metcalfe
Spatio-Temporal Methods in Environmental
Statistical Inference: An Integrated Approach,
Epidemiology
Second Edition
G. Shaddick and J.V. Zidek
H. S. Migon, D. Gamerman, and
Decision Analysis: A Bayesian Approach
F. Louzada
J.Q. Smith
Beyond ANOVA: Basics of Applied Statistics
Analysis of Failure and Survival Data
R.G. Miller, Jr.
P. J. Smith
A Primer on Linear Models
Applied Statistics: Handbook of GENSTAT
J.F. Monahan
Analyses
Stochastic Processes: From Applications to
E.J. Snell and H. Simpson
Theory
Applied Nonparametric Statistical Methods,
P.D Moral and S. Penev
Fourth Edition
Applied Stochastic Modelling, Second Edition
P. Sprent and N.C. Smeeton
B.J.T. Morgan
Data Driven Statistical Methods
Elements of Simulation
P. Sprent
B.J.T. Morgan
Generalized Linear Mixed Models:
Probability: Methods and Measurement
Modern Concepts, Methods and Applications
A. O’Hagan
W. W. Stroup
Introduction to Statistical Limit Theory
Survival Analysis Using S: Analysis of
A.M. Polansky
Time-to-Event Data
Applied Bayesian Forecasting and Time Series M. Tableman and J.S. Kim
Analysis
Applied Categorical and Count Data Analysis
A. Pole, M. West, and J. Harrison
W. Tang, H. He, and X.M. Tu
Statistics in Research and Development,
Elementary Applications of Probability Theory,
Time Series: Modeling, Computation, and
Second Edition
Inference
H.C. Tuckwell
R. Prado and M. West
Introduction to Statistical Inference and Its
Essentials of Probability Theory for
Applications with R
Statisticians
M.W. Trosset
M.A. Proschan and P.A. Shaw
Understanding Advanced Statistical Methods
Introduction to Statistical Process Control
P.H. Westfall and K.S.S. Henning
P. Qiu
Statistical Process Control: Theory and Epidemiology: Study Design and
Practice, Third Edition Data Analysis, Third Edition
G.B. Wetherill and D.W. Brown M. Woodward
Generalized Additive Models: Practical Data Analysis for Designed
An Introduction with R, Second Edition Experiments
S. Wood B.S. Yandell

Texts in Statistical Science
Statistical Regression
and Classification
From Linear Models to
Machine Learning
Norman Matloff
University of California, Davis, USA
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742

© 2017 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business

No claim to original U.S. Government works

Printed on acid-free paper

International Standard Book Number-13: 978-1-4987-1091-6 (Paperback)
International Standard Book Number-13: 978-1-138-06656-5 (Hardback)

This book contains information obtained from authentic and highly regarded sources. Reasonable efforts
have been made to publish reliable data and information, but the author and publisher cannot assume
responsibility for the validity of all materials or the consequences of their use. The authors and publishers
have attempted to trace the copyright holders of all material reproduced in this publication and apologize
to copyright holders if permission to publish in this form has not been obtained. If any copyright material
has not been acknowledged please write and let us know so we may rectify in any future reprint.

Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced,
transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or
hereafter invented, including photocopying, microfilming, and recording, or in any information storage
or retrieval system, without written permission from the publishers.

For permission to photocopy or use material electronically from this work, please access
www.copyright.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc.
(CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization
that provides licenses and registration for a variety of users. For organizations that have been granted a
photocopy license by the CCC, a separate system of payment has been arranged.

Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are
used only for identification and explanation without intent to infringe.

Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com

and the CRC Press Web site at
http://www.crcpress.com
Contents
Preface xxix
List of Symbols xxxvii
1 Setting the Stage 1
1.1 Example: Predicting Bike-Sharing
Activity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Example of the Prediction Goal: Body Fat . . . . . . . . . 2
1.3 Example: Who Clicks Web Ads? . . . . . . . . . . . . . . . 3
1.4 Approach to Prediction . . . . . . . . . . . . . . . . . . . . 4
1.5 A Note about E(), Samples and Populations . . . . . . . . . 5
1.6 Example of the Description Goal: Do
Baseball Players Gain Weight As They Age? . . . . . . . . 6
1.6.1 Prediction vs. Description . . . . . . . . . . . . . . . 7
1.6.2 A First Estimator . . . . . . . . . . . . . . . . . . . 9
1.6.3 A Possibly Better Estimator, Using a Linear Model 10
1.7 Parametric vs. Nonparametric Models . . . . . . . . . . . . 15
1.8 Example: Click-Through Rate. . . . . . . . . . . . . . . . . 15
1.9 Several Predictor Variables . . . . . . . . . . . . . . . . . . 17
1.9.1 Multipredictor Linear Models . . . . . . . . . . . . . 18
ix