Functional Data Analysis, Second Edition

432 Pages·2005·3.6 MB·english

by  Jim Ramsay

Springer Series in Statistics
Advisors:
P. Bickel, P. Diggle, S. Fienberg, U. Gather,
I. Olkin, S. Zeger
J.O. Ramsay
B.W. Silverman
Functional Data Analysis
Second Edition
With 151 Illustrations
J.O.Ramsay B.W.Silverman
DepartmentofPsychology St.Peter’sCollege
UniversityofMontreal OxfordOX12DL
Montreal,QuebecH3A1B1 UnitedKingdom
Canada [email protected]
[email protected]
LibraryofCongressControlNumber:2005923773
ISBN-10:0-387-40080-X Printedonacid-freepaper.
ISBN-13:978-0387-40080-8
©2005SpringerScience+BusinessMedia,Inc.
All rights reserved. This work may not be translated or copied in whole or in part without the
writtenpermission ofthepublisher (SpringerScience+BusinessMedia, Inc.,233Spring St.,New
York,NY10013,USA),exceptforbriefexcerptsinconnectionwithreviewsorscholarlyanalysis.
Useinconnectionwithanyformofinformationstorageandretrieval,electronicadaptation,com-
putersoftware,orbysimilarordissimilarmethodologynowknownorhereafterdevelopedisfor-
bidden.
The use in this publication of trade names, trademarks, service marks, and similar terms, even if
theyarenotidentifiedassuch,isnottobetakenasanexpressionofopinionastowhetherornot
theyaresubjecttoproprietaryrights.
PrintedintheUnitedStatesofAmerica. (MP)
9 8 7 6 5 4 3 2 1
springeronline.com
J.O.Ramsay B.W.Silverman
DepartmentofPsychology St.Peter’sCollege
McGillUniversity OxfordOX12DL
Montreal,QuebecH4A1B1 UnitedKingdom
Canada [email protected]
[email protected]
LibraryofCongressControlNumber:2005923773
ISBN-10:0-387-40080-X Printedonacid-freepaper.
ISBN-13:978-0387-40080-8
©2005SpringerScience+BusinessMedia,Inc.
All rights reserved. This work may not be translated or copied in whole or in part without the
writtenpermission ofthepublisher (SpringerScience+BusinessMedia, Inc.,233Spring St.,New
York,NY10013,USA),exceptforbriefexcerptsinconnectionwithreviewsorscholarlyanalysis.
Useinconnectionwithanyformofinformationstorageandretrieval,electronicadaptation,com-
putersoftware,orbysimilarordissimilarmethodologynowknownorhereafterdevelopedisfor-
bidden.
The use in this publication of trade names, trademarks, service marks, and similar terms, even if
theyarenotidentifiedassuch,isnottobetakenasanexpressionofopinionastowhetherornot
theyaresubjecttoproprietaryrights.
PrintedintheUnitedStatesofAmerica. (MP)
9 8 7 6 5 4 3 2 1
springeronline.com
Preface to the Second Edition
This book continues in the footsteps of the First Edition in being a snap-
shotofahighlysocial,andthereforedecidedlyunpredictable,process.The
combined personal view of functional data analysis that it presents has
emerged over a number of years of research and contact, and has been
greatly nourished by delightful collaborations with many friends. We hope
that readers will enjoy the book as much as we have enjoyed writing it,
whether they are our colleagues as researchers or applied data analysts
reading thebook asa research monograph, or studentsusing it asa course
text.
AsintheFirstEdition,livedataareusedthroughoutforbothmotivation
and illustration, showing how functional approaches allow us to see new
things, especially by exploiting the smoothness of the processes generating
thedata.Thedatasetsexemplifythewidescopeoffunctionaldataanalysis;
they are drawn from growth analysis, meteorology, biomechanics, equine
science, economics and medicine.
“Back to the data” was the heading to the last section of the First Edi-
tion. We did not know then how well those words would predict the next
eight years. Since then we have seen functional data applications in more
scientific and industrial settings than we could have imagined, and so we
wanted an opportunity to make this new field accessible to a wider reader-
ship than the the first volume seemed to permit. Our book of case studies,
Ramsay and Silverman (2002), was our first response, but we have known
for some time that a new edition of our original volume was also required.
We have added a considerable amount of new material, and considered
carefully how the original material should be presented. One main objec-
vi
tivehasbeen,especiallywhenintroducingthevariousconcepts,toprovide
moreintuitivediscussionandtopostponeneedlessmathematicalterminol-
ogy where possible. In addition we wanted to offer more practical advice
on the processing of functional data. To this end, we have added a more
extended account of spline basis functions, provided new material on data
smoothing, and extended the range of ways in which data can be used
to estimate functions. In response to many requests, we have added some
proposalsforestimatingconfidenceregions,highlightinglocalfeatures,and
eventestinghypotheses.Nevertheless,theemphasisintherevisionremains
more exploratory and confirmatory.
Our treatment of the functional linear model in the First Edition was
only preliminary, and since then a great deal of work has been done on
this topic by many investigators. A complete overhaul of this material was
called for, and the chapters on linear modelling have been completely re-
worked. On the other hand, our coverage of principal components analysis
and canonical correlation still seems appropriate, and not much has been
changed. Readers reacted to the later chapters on differential equations as
being difficult, and so we have tried to make them a friendlier place to be.
In some places we have opted for an ‘intuitive’ rather than ‘rigorous’
approach.Thisisnotmerelybecausewewantourbooktobewidelyacces-
sible; in our view the theoretical underpinnings of functional data analysis
still require rather more study before a treatment can be written that will
please theoreticians. We hope that the next decade will see some exciting
progress in this regard.
Webothbelievethatagoodmonographisapersonalviewratherthana
dry encyclopedia. The average of two personal views is inevitably going to
beless‘personal’thaneitherofthetwoindividualviews,justastheaverage
of a set of functions may omit detail present in the original functions. To
counteract this tendency, we have ensured that everything we say in our
informalandintuitivediscussionofcertainissuesistheviewofatleastone
of us, but we have not always pressed for unanimous agreement!
We owe so much to those who helped us to go here. We would like to
repeatourthankstothosewhohelpedwiththeFirstEdition:MichalAbra-
hamowicz, Philippe Besse, Darrell Bock, Catherine Dalzell, Shelly Feran,
Randy Flanagan, Rowena Fowler, Theo Gasser, Mary Gauthier, Vince
Gracco,NancyHeckman,AnoukHoedeman,SteveHunka,IainJohnstone,
Alois Kneip, Wojtek Krzanowski, Xiaochun Li, Kevin Munhall, Guy Na-
son, Richard Olshen, David Ostry, Tim Ramsay, John Rice and Xiaohui
Wang.Wealsocontinueourgratefulacknowledgementoffinancialsupport
fromtheNaturalScienceandEngineeringResearchCouncilofCanada,the
National Science Foundation and the National Institute of Health of the
USA,andtheBritishEngineeringandPhysicalSciencesResearchCouncil.
The seed for the First Edition, and therefore for the Revised Edition as
well, was planted at a discussion meeting of the Royal Statistical Society
vii
ResearchSection,whereoneofusreadapaperandtheotherproposedthe
vote of thanks, not always an occasion that leads to a meeting of minds!
Turning to the Second Edition, Sofia Mosesova and Yoshio Takane read
theentiremanuscriptwithaneyetothetechnicalcorrectnessaswellasthe
readabilityofwhattheysaw,andcaughtusonmanypoints.DavidCamp-
bellhelpedwiththeliteraturereviewthatsupportedour“Furtherreadings
andnotes”sections.TimespentattheUniversityofBritishColumbiamade
possiblemanystimulatingconversationswithNancyHeckmanandhercol-
leagues.AdiscussionofmanyissueswithAloisKneipaswellhishospitality
for the first author at the University of Mainz was invaluable. The oppor-
tunity for ustospend timetogether afforded by St Peter’sCollegeand the
Department of Statistics at Oxford University was essential to the project.
April 2005 Jim Ramsay & Bernard Silverman
Contents
Preface to the Second Edition v
1 Introduction 1
1.1 What are functional data? . . . . . . . . . . . . . . . . . 1
1.2 Functional models for nonfunctional data . . . . . . . . . 5
1.3 Some functional data analyses . . . . . . . . . . . . . . . 5
1.4 The goals of functional data analysis . . . . . . . . . . . 9
1.5 The first steps in a functional data analysis. . . . . . . . 11
1.5.1 Data representation: smoothing and interpolation 11
1.5.2 Data registration or feature alignment . . . . . . 12
1.5.3 Data display . . . . . . . . . . . . . . . . . . . . . 13
1.5.4 Plotting pairs of derivatives . . . . . . . . . . . . 13
1.6 Exploring variability in functional data . . . . . . . . . . 15
1.6.1 Functional descriptive statistics . . . . . . . . . . 15
1.6.2 Functional principal components analysis . . . . . 15
1.6.3 Functional canonical correlation . . . . . . . . . . 16
1.7 Functional linear models . . . . . . . . . . . . . . . . . . 16
1.8 Using derivatives in functional data analysis . . . . . . . 17
1.9 Concluding remarks . . . . . . . . . . . . . . . . . . . . . 18
2 Tools for exploring functional data 19
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.2 Some notation . . . . . . . . . . . . . . . . . . . . . . . . 20
2.2.1 Scalars, vectors, functions and matrices . . . . . . 20
x Contents
2.2.2 Derivatives and integrals . . . . . . . . . . . . . . 20
2.2.3 Inner products. . . . . . . . . . . . . . . . . . . . 21
2.2.4 Functions of functions . . . . . . . . . . . . . . . 21
2.3 Summary statistics for functional data . . . . . . . . . . 22
2.3.1 Functional means and variances . . . . . . . . . . 22
2.3.2 Covariance and correlation functions . . . . . . . 22
2.3.3 Cross-covariance and cross-correlation functions . 24
2.4 The anatomy of a function . . . . . . . . . . . . . . . . . 26
2.4.1 Functional features . . . . . . . . . . . . . . . . . 26
2.4.2 Data resolution and functional dimensionality . . 27
2.4.3 The size of a function . . . . . . . . . . . . . . . . 28
2.5 Phase-plane plots of periodic effects . . . . . . . . . . . . 29
2.5.1 The log nondurable goods index . . . . . . . . . . 29
2.5.2 Phase–plane plots show energy transfer . . . . . . 30
2.5.3 The nondurable goods cycles . . . . . . . . . . . . 33
2.6 Further reading and notes . . . . . . . . . . . . . . . . . 34
3 From functional data to smooth functions 37
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.2 Some properties of functional data . . . . . . . . . . . . 38
3.2.1 What makes discrete data functional?. . . . . . . 38
3.2.2 Samples of functional data . . . . . . . . . . . . . 39
3.2.3 The interplay between smooth and noisy variation 39
3.2.4 The standard model for error and its limitations . 40
3.2.5 The resolving power of data . . . . . . . . . . . . 41
3.2.6 Data resolution and derivative estimation . . . . 41
3.3 Representing functions by basis functions . . . . . . . . . 43
3.4 The Fourier basis system for periodic data . . . . . . . . 45
3.5 The spline basis system for open-ended data . . . . . . . 46
3.5.1 Spline functions and degrees of freedom. . . . . . 47
3.5.2 The B-spline basis for spline functions . . . . . . 49
3.6 Other useful basis systems . . . . . . . . . . . . . . . . . 53
3.6.1 Wavelets . . . . . . . . . . . . . . . . . . . . . . . 53
3.6.2 Exponential and power bases . . . . . . . . . . . 54
3.6.3 Polynomial bases . . . . . . . . . . . . . . . . . . 54
3.6.4 The polygonal basis . . . . . . . . . . . . . . . . . 55
3.6.5 The step-function basis . . . . . . . . . . . . . . . 55
3.6.6 The constant basis . . . . . . . . . . . . . . . . . 55
3.6.7 Empirical and designer bases. . . . . . . . . . . . 56
3.7 Choosing a scale for t . . . . . . . . . . . . . . . . . . . . 56
3.8 Further reading and notes . . . . . . . . . . . . . . . . . 57
4 Smoothing functional data by least squares 59
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 59
4.2 Fitting data using a basis system by least squares . . . . 59
Contents xi
4.2.1 Ordinary or unweighted least squares fits . . . . . 60
4.2.2 Weighted least squares fits . . . . . . . . . . . . . 61
4.3 A performance assessment of least squares smoothing . . 62
4.4 Least squares fits as linear transformations of the data . 63
4.4.1 How linear smoothers work. . . . . . . . . . . . . 64
4.4.2 The degrees of freedom of a linear smooth . . . . 66
4.5 Choosing the number K of basis functions . . . . . . . . 67
4.5.1 The bias/variance trade-off. . . . . . . . . . . . . 67
4.5.2 Algorithms for choosing K . . . . . . . . . . . . . 69
4.6 Computing sampling variances and confidence limits . . 70
4.6.1 Sampling variance estimates . . . . . . . . . . . . 70
4.6.2 Estimating Σ . . . . . . . . . . . . . . . . . . . . 71
e
4.6.3 Confidence limits . . . . . . . . . . . . . . . . . . 72
4.7 Fitting data by localized least squares. . . . . . . . . . . 73
4.7.1 Kernel smoothing . . . . . . . . . . . . . . . . . . 74
4.7.2 Localized basis function estimators . . . . . . . . 76
4.7.3 Local polynomial smoothing . . . . . . . . . . . . 77
4.7.4 Choosing the bandwidth h . . . . . . . . . . . . . 78
4.7.5 Summary of localized basis methods . . . . . . . 78
4.8 Further reading and notes . . . . . . . . . . . . . . . . . 79
5 Smoothing functional data with a roughness penalty 81
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 81
5.2 Spline smoothing . . . . . . . . . . . . . . . . . . . . . . 82
5.2.1 Two competing objectives in function estimation 83
5.2.2 Quantifying roughness . . . . . . . . . . . . . . . 84
5.2.3 Thepenalizedsumofsquarederrorsfittingcriterion 84
5.2.4 The structure of a smoothing spline . . . . . . . . 85
5.2.5 How spline smooths are computed . . . . . . . . . 86
5.2.6 Spline smoothing as a linear operation . . . . . . 87
5.2.7 Spline smoothing as an augmented least squares
problem . . . . . . . . . . . . . . . . . . . . . . . 89
5.2.8 Estimating derivatives by spline smoothing . . . . 90
5.3 Some extensions . . . . . . . . . . . . . . . . . . . . . . . 91
5.3.1 Roughness penalties with fewer basis functions. . 91
5.3.2 More general measures of data fit . . . . . . . . . 92
5.3.3 More general roughness penalties . . . . . . . . . 92
5.3.4 Computing the roughness penalty matrix . . . . . 93
5.4 Choosing the smoothing parameter . . . . . . . . . . . . 94
5.4.1 Some limits imposed by computational issues . . 94
5.4.2 The cross-validation or CV method . . . . . . . . 96
5.4.3 The generalized cross-validation or GCV method 97
5.4.4 Spline smoothing the simulated growth data . . . 99
5.5 Confidence intervals for function values and functional
probes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100