Linear models with R

284 Pages·2015·47.625 MB·English

by  Faraway, Julian James

Statistics Second
Texts in Statistical Science
Edition
Part of the core of statistics, linear models are used to make predictions
and explain the relationship between the response and the predictors.
Understanding linear models is crucial to a broader competence in the L Linear Models
practice of statistics. Linear Models with R, Second Edition explains how i
to use linear models in physical science, engineering, social science, and n
business applications. The book incorporates several improvements that e
with R
reflect how the world of R has greatly expanded since the publication of a
the first edition.
r

New to the Second Edition M
• Reorganized material on interpreting linear models, which Second Edition
distinguishes the main applications of prediction and explanation and o
introduces elementary notions of causality d
• Additional topics, including QR decomposition, splines, additive e
models, Lasso, multiple imputation, and false discovery rates
l
• Extensive use of the ggplot2 graphics package in addition to base s
graphics
w
Like its widely praised, best-selling predecessor, this edition combines
i
statistics and R to seamlessly give a coherent exposition of the practice of
t
linear modeling. The text offers up-to-date insight on essential data analysis h
topics, from estimation, inference, and prediction to missing data, factorial
R
models, and block designs. Numerous examples illustrate how to apply the
different methods using R.
Features
• Demonstrates the flexibility of linear models in many examples
• Assumes basic knowledge of R and statistics
• Emphasizes intuition over rigorous proofs
• Presents exercises at the end of each chapter
• Includes datasets and R commands
F
a
r
a Julian J. Faraway
w
a
y
K14039
Linear Models
with R
Second Edition
CHAPMAN & HALL/CRC
Texts in Statistical Science Series
Series Editors
Francesca Dominici, Harvard School of Public Health, USA
Julian J. Faraway, University of Bath, UK
Martin Tanner, Northwestern University, USA
Jim Zidek, University of British Columbia, Canada
Statistical Theory: A Concise Introduction Introduction to Statistical Methods for
F. Abramovich and Y. Ritov Clinical Trials
T.D. Cook and D.L. DeMets
Practical Multivariate Analysis, Fifth Edition
A. Afifi, S. May, and V.A. Clark Applied Statistics: Principles and Examples
Practical Statistics for Medical Research D.R. Cox and E.J. Snell
D.G. Altman Multivariate Survival Analysis and Competing
Interpreting Data: A First Course Risks
in Statistics M. Crowder
A.J.B. Anderson Statistical Analysis of Reliability Data
Introduction to Probability with R M.J. Crowder, A.C. Kimber,
K. Baclawski T.J. Sweeting, and R.L. Smith
Linear Algebra and Matrix Analysis for An Introduction to Generalized
Statistics Linear Models, Third Edition
S. Banerjee and A. Roy A.J. Dobson and A.G. Barnett
Statistical Methods for SPC and TQM Nonlinear Time Series: Theory, Methods, and
D. Bissell Applications with R Examples
R. Douc, E. Moulines, and D.S. Stoffer
Bayesian Methods for Data Analysis,
Third Edition Introduction to Optimization Methods and
B.P. Carlin and T.A. Louis Their Applications in Statistics
B.S. Everitt
Second Edition
R. Caulcutt Extending the Linear Model with R:
Generalized Linear, Mixed Effects and
The Analysis of Time Series: An Introduction,
Nonparametric Regression Models
Sixth Edition
J.J. Faraway
C. Chatfield
Linear Models with R, Second Edition
Introduction to Multivariate Analysis
J.J. Faraway
C. Chatfield and A.J. Collins
A Course in Large Sample Theory
Problem Solving: A Statistician’s Guide,
T.S. Ferguson
Second Edition
C. Chatfield Multivariate Statistics: A Practical
Approach
Statistics for Technology: A Course in Applied
B. Flury and H. Riedwyl
Statistics, Third Edition
C. Chatfield Readings in Decision Analysis
S. French
Bayesian Ideas and Data Analysis: An
Introduction for Scientists and Statisticians Markov Chain Monte Carlo:
Stochastic Simulation for Bayesian Inference,
R. Christensen, W. Johnson, A. Branscum,
Second Edition
and T.E. Hanson
D. Gamerman and H.F. Lopes
Modelling Binary Data, Second Edition
Bayesian Data Analysis, Third Edition
D. Collett
A. Gelman, J.B. Carlin, H.S. Stern, D.B. Dunson,
Modelling Survival Data in Medical Research,
A. Vehtari, and D.B. Rubin
Second Edition
D. Collett
Multivariate Analysis of Variance and Statistical Theory, Fourth Edition
Repeated Measures: A Practical Approach for B.W. Lindgren
Behavioural Scientists
Stationary Stochastic Processes: Theory and
D.J. Hand and C.C. Taylor
Applications
Practical Data Analysis for Designed Practical G. Lindgren
Longitudinal Data Analysis
The BUGS Book: A Practical Introduction to
D.J. Hand and M. Crowder
Bayesian Analysis
Logistic Regression Models D. Lunn, C. Jackson, N. Best, A. Thomas, and
J.M. Hilbe D. Spiegelhalter
Richly Parameterized Linear Models: Introduction to General and Generalized
Additive, Time Series, and Spatial Models Linear Models
Using Random Effects H. Madsen and P. Thyregod
J.S. Hodges
Time Series Analysis
Statistics for Epidemiology H. Madsen
N.P. Jewell
Pólya Urn Models
Stochastic Processes: An Introduction, H. Mahmoud
Second Edition
Randomization, Bootstrap and Monte Carlo
P.W. Jones and P. Smith
Methods in Biology, Third Edition
The Theory of Linear Models B.F.J. Manly
B. Jørgensen
Introduction to Randomized Controlled
Principles of Uncertainty Clinical Trials, Second Edition
J.B. Kadane J.N.S. Matthews
Graphics for Statistics and Data Analysis with R Statistical Methods in Agriculture and
K.J. Keen Experimental Biology, Second Edition
R. Mead, R.N. Curnow, and A.M. Hasted
Mathematical Statistics
K. Knight Statistics in Engineering: A Practical Approach
A.V. Metcalfe
Introduction to Multivariate Analysis:
Linear and Nonlinear Modeling Beyond ANOVA: Basics of Applied Statistics
S. Konishi R.G. Miller, Jr.
Nonparametric Methods in Statistics with SAS A Primer on Linear Models
Applications J.F. Monahan
O. Korosteleva
Applied Stochastic Modelling, Second Edition
Modeling and Analysis of Stochastic Systems, B.J.T. Morgan
Second Edition
Elements of Simulation
V.G. Kulkarni
B.J.T. Morgan
Exercises and Solutions in Biostatistical Theory
Probability: Methods and Measurement
L.L. Kupper, B.H. Neelon, and S.M. O’Brien
A. O’Hagan
Exercises and Solutions in Statistical Theory
Introduction to Statistical Limit Theory
L.L. Kupper, B.H. Neelon, and S.M. O’Brien
A.M. Polansky
Design and Analysis of Experiments with SAS
Applied Bayesian Forecasting and Time Series
J. Lawson
Analysis
A Course in Categorical Data Analysis A. Pole, M. West, and J. Harrison
T. Leonard
Statistics in Research and Development,
Statistics for Accountants Time Series: Modeling, Computation, and
S. Letchford Inference
Introduction to the Theory of Statistical R. Prado and M. West
Inference Introduction to Statistical Process Control
H. Liero and S. Zwanzig P. Qiu
Sampling Methodologies with Applications Generalized Linear Mixed Models:
P.S.R.S. Rao Modern Concepts, Methods and Applications
W. W. Stroup
A First Course in Linear Model Theory
N. Ravishanker and D.K. Dey Survival Analysis Using S: Analysis of
Time-to-Event Data
Essential Statistics, Fourth Edition
M. Tableman and J.S. Kim
D.A.G. Rees
Applied Categorical and Count Data Analysis
Stochastic Modeling and Mathematical
W. Tang, H. He, and X.M. Tu
Statistics: A Text for Statisticians and
Quantitative Elementary Applications of Probability Theory,
F.J. Samaniego Second Edition
H.C. Tuckwell
Statistical Methods for Spatial Data Analysis
O. Schabenberger and C.A. Gotway Introduction to Statistical Inference and Its
Applications with R
Large Sample Methods in Statistics
M.W. Trosset
P.K. Sen and J. da Motta Singer
Understanding Advanced Statistical Methods
Decision Analysis: A Bayesian Approach
P.H. Westfall and K.S.S. Henning
J.Q. Smith
Statistical Process Control: Theory and
Analysis of Failure and Survival Data
Practice, Third Edition
P. J. Smith
G.B. Wetherill and D.W. Brown
Applied Statistics: Handbook of GENSTAT
Generalized Additive Models:
Analyses
An Introduction with R
E.J. Snell and H. Simpson
S. Wood
Applied Nonparametric Statistical Methods,
Epidemiology: Study Design and
Fourth Edition
Data Analysis, Third Edition
P. Sprent and N.C. Smeeton
M. Woodward
Data Driven Statistical Methods
Experiments
P. Sprent
B.S. Yandell
Texts in Statistical Science
Linear Models
with R
Second Edition
Julian J. Faraway
University of Bath
United Kingdom
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2014 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Version Date: 20160223
International Standard Book Number-13: 978-1-4398-8734-9 (eBook - PDF)
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts
have been made to publish reliable data and information, but the author and publisher cannot assume
responsibility for the validity of all materials or the consequences of their use. The authors and publishers
have attempted to trace the copyright holders of all material reproduced in this publication and apologize to
copyright holders if permission to publish in this form has not been obtained. If any copyright material has
not been acknowledged please write and let us know so we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmit-
ted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented,
including photocopying, microfilming, and recording, or in any information storage or retrieval system,
without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright.
com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood
Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and
registration for a variety of users. For organizations that have been granted a photocopy license by the CCC,
a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used
only for identification and explanation without intent to infringe.
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
Contents
Preface xi
1 Introduction 1
1.1 BeforeYouStart 1
1.2 InitialDataAnalysis 2
1.3 WhentoUseLinearModeling 7
1.4 History 8
2 Estimation 13
2.1 LinearModel 13
2.2 MatrixRepresentation 14
2.3 Estimatingβ 15
2.4 LeastSquaresEstimation 16
2.5 ExamplesofCalculatingβˆ 17
2.6 Example 17
2.7 QRDecomposition 20
2.8 Gauss–MarkovTheorem 22
2.9 GoodnessofFit 23
2.10 Identifiability 26
2.11 Orthogonality 28
3 Inference 33
3.1 HypothesisTeststoCompareModels 33
3.2 TestingExamples 35
3.3 PermutationTests 40
3.4 Sampling 42
3.5 ConfidenceIntervalsforβ 43
3.6 BootstrapConfidenceIntervals 46
4 Prediction 51
4.1 ConfidenceIntervalsforPredictions 51
4.2 PredictingBodyFat 52
4.3 Autoregression 54
4.4 WhatCanGoWrongwithPredictions? 56
vii
viii CONTENTS
5 Explanation 59
5.1 SimpleMeaning 59
5.2 Causality 61
5.3 DesignedExperiments 62
5.4 ObservationalData 63
5.5 Matching 65
5.6 CovariateAdjustment 68
5.7 QualitativeSupportforCausation 69
6 Diagnostics 73
6.1 CheckingErrorAssumptions 73
6.1.1 ConstantVariance 73
6.1.2 Normality 78
6.1.3 CorrelatedErrors 81
6.2 FindingUnusualObservations 83
6.2.1 Leverage 83
6.2.2 Outliers 85
6.2.3 InfluentialObservations 89
6.3 CheckingtheStructureoftheModel 92
6.4 Discussion 96
7 ProblemswiththePredictors 99
7.1 ErrorsinthePredictors 99
7.2 ChangesofScale 103
7.3 Collinearity 106
8 ProblemswiththeError 113
8.1 GeneralizedLeastSquares 113
8.2 WeightedLeastSquares 116
8.3 TestingforLackofFit 119
8.4 RobustRegression 123
8.4.1 M-Estimation 123
8.4.2 LeastTrimmedSquares 126
9 Transformation 133
9.1 TransformingtheResponse 133
9.2 TransformingthePredictors 137
9.3 BrokenStickRegression 137
9.4 Polynomials 139
9.5 Splines 141
9.6 AdditiveModels 144
9.7 MoreComplexModels 145
CONTENTS ix
10 ModelSelection 149
10.1 HierarchicalModels 150
10.2 Testing-BasedProcedures 151
10.3 Criterion-BasedProcedures 153
10.4 Summary 159
11 ShrinkageMethods 161
11.1 PrincipalComponents 161
11.2 PartialLeastSquares 172
11.3 RidgeRegression 174
11.4 Lasso 177
12 InsuranceRedlining—ACompleteExample 183
12.1 EcologicalCorrelation 183
12.2 InitialDataAnalysis 185
12.3 FullModelandDiagnostics 188
12.4 SensitivityAnalysis 190
12.5 Discussion 193
13 MissingData 197
13.1 TypesofMissingData 197
13.2 Deletion 198
13.3 SingleImputation 200
13.4 MultipleImputation 202
14 CategoricalPredictors 205
14.1 ATwo-LevelFactor 205
14.2 FactorsandQuantitativePredictors 209
14.3 InterpretationwithInteractionTerms 212
14.4 FactorsWithMoreThanTwoLevels 213
14.5 AlternativeCodingsofQualitativePredictors 219
15 OneFactorModels 223
15.1 TheModel 223
15.2 AnExample 224
15.3 Diagnostics 227
15.4 PairwiseComparisons 228
15.5 FalseDiscoveryRate 230
16 ModelswithSeveralFactors 235
16.1 TwoFactorswithNoReplication 235
16.2 TwoFactorswithReplication 239
16.3 TwoFactorswithanInteraction 243
16.4 LargerFactorialExperiments 246