Asymptotic theory of statistics and probability

747 Pages·2008·4.028 MB·English

by  Anirban DasGupta (auth.)

Springer Texts in Statistics
Series Editors:
G. Casella
S. Fienberg
I. Olkin
Springer Texts in Statistics
Athreya/Lahiri: Measure Theory and Probability Theory
Bilodeau/Brenner: Theory of Multivariate Statistics
Brockwell/Davis: An Introduction to Time Series and Forecasting
Carmona: Statistical Analysis of Financial Data in S-PLUS
Casella: Statistical Design
Chow/Teicher: Probability Theory: Independence, Interchangeability, Martingales,
Third Edition
Christensen: Advanced Linear Modeling: Multivariate, Time Series, and Spatial Data;
Nonparametric Regression and Response Surface Maximization, Second Edition
Christensen: Log-Linear Models and Logistic Regression, Second Edition
Christensen: Plane Answers to Complex Questions: The Theory of Linear Models,
Second Edition
Cryer/Chan: Time Series Analysis, Second Edition
DasGupta: Asymptotic Theory of Statistics and Probability
Davis: Statistical Methods for the Analysis of Repeated Measurements
Dean/Voss: Design and Analysis of Experiments
Dekking/Kraaikamp/Lopuhaä/Meester: A Modern Introduction to Probability
and Statistics
Durrett: Essentials of Stochastic Processes
Edwards: Introduction to Graphical Modeling, Second Edition
Everitt: An R and S-PLUS Companion to Multivariate Analysis
Gentle: Matrix Algebra: Theory, Computations, and Applications in Statistics
Ghosh/Delampady/Samanta: An Introduction to Bayesian Analysis
Gut: Probability: A Graduate Course
Heiberger/Holland: Statistical Analysis and Data Display; An Intermediate Course
with Examples in S-PLUS, R, and SAS
Jobson: Applied Multivariate Data Analysis, Volume I: Regression
and Experimental Design
Jobson: Applied Multivariate Data Analysis, Volume II: Categorical
and Multivariate Methods
Karr: Probability
Kulkarni: Modeling, Analysis, Design, and Control of Stochastic Systems
Lange: Applied Probability
Lange: Optimization
Lehmann: Elements of Large Sample Theory
Lehmann/Romano: Testing Statistical Hypotheses, Third Edition
Lehmann/Casella: Theory of Point Estimation, Second Edition
Longford: Studying Human Populations: An Advanced Course in Statistics
Marin/Robert: Bayesian Core: A Practical Approach to Computational
Bayesian Statistics
Nolan/Speed: Stat Labs: Mathematical Statistics Through Applications
Pitman: Probability
Rawlings/Pantula/Dickey: Applied Regression Analysis
(continued after index)
Anirban DasGupta
Asymptotic Theory
of Statistics and Probability
Anirban DasGupta
Purdue University
Department of Statistics
150 North University Street
West Lafayette, IN 47907
[email protected]
Editorial Board
George Casella Stephen Fienberg Ingram Olkin
Department of Statistics Department of Statistics Department of Statistics
University of Florida Carnegie Mellon University Stanford University
Gainesville, FL 32611-8545 Pittsburgh, PA 15213-3890 Stanford, CA 94305
USA USA USA
ISBN 978-0-387-75970-8 e-ISBN 978-0-387-75971-5
DOI: 10.1007/978-0-387-75971-5
Library of Congress Control Number: 2008921241
© 2008 Springer Science+Business Media, LLC
All rights reserved. This work may not be translated or copied in whole or in part without the written
permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York,
NY, 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use
in connection with any form of information storage and retrieval, electronic adaptation, computer
software, or by similar or dissimilar methodology now known or hereafter developed is forbidden.
The use in this publication of trade names, trademarks, service marks, and similar terms, even if they
are not identifi ed as such, is not to be taken as an expression of opinion as to whether or not they are
subject to proprietary rights.
Printed on acid-free paper.
9 8 7 6 5 4 3 2 1
springer.com
To mymother,andtothelovingmemories
ofmyfather
Preface
This book developed out of my year-long course on asymptotic theory at
PurdueUniversity.Tosomeextent,the topicscoincidewith whatIcoverin
that course. There are already a number of well-known books on asymp-
totics.This bookis quite different.It coversmoretopicsin onesourcethan
areavailableinanyothersinglebookonasymptotictheory.Numeroustopics
covered in this book are available in the literature in a scattered manner,
andthey are broughttogetherunderone umbrella in this book.Asymptotic
theory is a central unifying theme in probability and statistics. My main
goalinwriting thisbookistogiveitsreadersafeelfortheincrediblescope
and reach of asymptotics. I have tried to write this book in a way that is
accessible and to make the reader appreciate the beauty of theory and the
insightsthatonlytheorycanprovide.
Essentiallyeverytheoremin the bookcomeswith atleastonereference,
precedingorfollowingthestatementofthetheorem.Inaddition,Ihavepro-
videdaseparatetheorem-by-theoremreferenceasanentryonitsowninthe
front of the book to make it extremely convenient for the reader to find a
proof thatwas not providedin the text. Also particularly worth mentioning
is a collection of nearly 300 practically useful inequalities that I have col-
lected together from numerous sources. This is appended at the very end
of the book. Almost every inequality in this collection comes with at least
one reference. I have often preferred to cite a book rather than an original
publication for these inequalities, particularly if the book contained many
of the inequalities that I present. I also emphasize in this book conceptual
discussionofissues,workingoutmanyexamplesandprovidingagoodcol-
lectionofunusualexercises.Anotherfeatureofthisbookistheguidanceto
theliteratureforsomeonewhowishestodigdeeperintothetopicofapartic-
ularchapter.Ihavetriedtomakethechapter-by-chapterbibliographiesboth
modernandrepresentative.Thebookhas574exercisesand293worked-out
examples.Ihavemarkedthemorenonroutineexerciseswithanasterisk.
vii
viii Preface
Ihopethatthisbookisusefulasagraduatetext,forindependentreading,
and as a general and nearly encyclopedicresearch reference on asymptotic
theory.Itshouldbepossibletodesigngraduate-levelcoursesusingthisbook
withemphasisonparametricmethodsornonparametricmethods,onclassic
topics or more current topics, on frequentist topics or Bayesian topics, or
evenonprobabilitytheory.Forthebenefitofinstructors,Ihaveprovidedrec-
ommended chapters for ten different one-semester courses, with emphasis
on different themes. I hope that this provides some useful guidancetoward
designingcoursesbasedonthisbook.
Because the book covers a very broad range of topics, I do not have a
uniform answer for what background I assume for a reader of this book.
For most chapters, a knowledge of advancedcalculus and linear algebra is
enoughtoenablethereadertofollowthematerial.However,somechapters
require someuseof measuretheory andadvancedanalysisandsomeexpo-
sure to stochastic processes.One course on statistical theory at the level of
BickelandDoksum(citedinChapter3ofthisvolume)orCasellaandBerger
(1990) and one on probability at the level of Hoel, Port, and Stone (1971)
orDurrett(1994)arecertainlyneededtofollowthediscussioninthisbook.
Chapter 1 is essentially a review of somewhat more advanced probability
should one need it. The more advanced chapters in this book can be much
betterappreciatedifonehashadcoursesonthetwobooksofErichLehmann
(Lehmann and Casella (cited in Chapter 16), Lehmann and Romano (cited
in Chapter 24)) and a course based on Breiman (1992), Durrett (2004) or
Billingsley(seeChapter1).
MygreatestthanksareduetoPeterHallforbeinganinspirationalandcar-
ingadvisor,reader,andintellectualfilteroverthelastseveralyearsasIwas
writing drafts of this book. Peter has deeply influenced my understanding,
appreciation,andtaste forprobability andstatistics, andIhavefelt grateful
thatIhavehadaccesstohimatalltimesandwithunlimitedpatience.Ihave
received much more from Peter than I could wish or expect. I could not
have written this book without Peter’s exemplary warmth and mentorship.
However, all mistakes and ambiguities in the book are exclusively my re-
sponsibility.Iwouldlovetoknowofallseriousmistakesthatareaderfinds
inthisbook,andtheremustbemistakesinabookofthislength.
I alsowant to expressmy very specialthanksto JohnMarden and Larry
Wassermanforrepeatedlyofferingtheirfriendlyandthoughtfulsuggestions
onvariousdecisionsIhadtomakeonthisbook.Iwanttomentionthegener-
oushelpandsupportfromErichLehmann,PeterBickel,RabiBhattacharya,
andJonWellneronspecificchaptersinthebook.Numerouscolleagues,and
inparticularC.R.Rao,ArupBose,PersiDiaconis,JoeEaton,JianqingFan,
Iain Johnstone, T. Krishnan, Bruce Lindsay, Wei-Liem Loh, Peter McCul-
lagh,Dimitris Politis, B.V.Rao,BobSerfling,J.Sethuraman,KesarSingh,
Preface ix
and Michael Woodroofe, made helpful comments on parts of earlier drafts
of the book. Chun Han, Martina Muehlbach, and Surya Tokdar helped me
graciouslywithputtingtogetherTeXfilesofthechapers.JohnKimmeland
JeffreyTaubatSpringerwereextremelyhelpfulandprofessional,andIen-
joyed working with them very much. I will work with Johnand Jeff at any
timewithpleasure.Severalanonymousrefereesdidunbelievablyhelpfuland
constructivereadingsofmanypartsofthebook.TheSpringerserieseditors
gavemegraciousinputwheneverneeded.ThecopyeditorHalHengleinand
the typesetters – Integra India of Springer did a magnificent job. I am im-
menselythankfultoallofthem.IamalsothankfultoPurdueUniversityfor
providing me with computing and secretarial assistance. Doug and Cheryl
Crabill,inparticular,assistedmenumeroustimeswithasmile.
I was an impressionable fifteen-year-old when I entered the Indian Sta-
tisticalInstitute (ISI) asafirst-yearstudent.Ihadheardthatstatisticiansdo
boringcalculationswith large numbersusingclumsycalculatingmachines.
Dev Basu entered the lecture room on my first day at the ISI and instantly
changed my perception of statistics. No one I met could explain so effort-
lessly the study of randomness and how to use what we learn about it to
make useful conclusions. There was not one person at the ISI who didn’t
regardhimasanincomparablerolemodel,apersonificationofscholarship,
and an angelic personality. I am fortunate that I had him as my foremost
teacher. I am grateful to C. R. Rao for the golden days of the ISI and for
makingallofusfeelthatevenasstudentswewereequalsinhiseyes.
At a personal level, I am profoundly grateful to Jenifer Brown for the
uniquenessandconstancyofhertreasuredsupport,counsel,wellwishes,and
endearingcamaraderieformanyyears,allofwhichhaveenrichedmeatmy
mostdifficulttimesandhavehelpedmebecomeabetterhumanbeing.Iwill
always remain much indebted to Jenifer for the positive, encouraging, and
crystallizinginfluenceshehasbeenatalltimes.IhaveconsideredJeniferto
beanimpeccablerolemodel.
I am also thankful to Supriyo and Anuradha Datta, Julie Marshburn,
Teena Seele, Gail Hytner, Norma Lucas, Deb Green, Tanya Winder, Hira
Koul,RajeevaKarandikar,Wei-LiemLoh,DimitrisPolitis,andLarryShepp
for their loyalty, friendship and warmth. Jim and Ann Berger, Herman Ru-
bin, B.V. Rao, T. Krishnan, Larry Brown, Len Haff, Jianqing Fan, and Bill
Strawderman have mentored, supported and cared for me for more than a
decade. I appreciate all of them. But most of all, I appreciate the love and
warmthofmyfamily.Idedicatethisbooktothecherishedmemoriesofmy
father,andtomymotheronhereightiethbirthday.
AnirbanDasGupta
PurdueUniversity,WestLafayette,IN
Recommended Chapter Selections
CourseType Chapters
SemesterI,Classical 1,2,3,4,7,8,11,13,15,17,21,26,27
Asymptotics
SemesterII,Classical 9,14,16,22,24,25,28,29,30,31,32
Asymptotics
SemesterI,Inference 1,2,3,4,7,14,16,17,19,20,21,26,27
SemesterII,Inference 8,11,12,13,22,24,25,29,30,32,33,34
Semester I, Emphasis on 1,2,3,4,5,6,8,9,10,11,12,23
Probability
Semester I, Contemporary 1,2,3,8,10,12,14,29,30,32,33,34
Topics
SemesterI, 1,3,5,7,11,13,15,18,24,26,29,30,32
Nonparametrics
Semester I, Modeling and 1,3,4,8,9,10,16,19,26,27,29,32,33
DataAnalysis
MyUsualCourse, 1,2,3,4,6,7,8,11,13,14,15,16,20
SemesterI
MyUsualCourse, 5,9,12,17,21,22,24,26,28,29,30,32,34
SemesterII
xi
Key Theorems and References
TheoremNumber ASourceforProof
1.2 Breiman(1968,1992)
1.4 Billingsley(1995)
1.5 Ferguson(1996)
1.6 Breiman(1968,1992)
1.7 Ash(1973),Billingsley(1995)
1.11 Breiman(1968,1992)
1.13 Billingsley(1995)
1.15 Breiman(1968,1992)
2.1 Reiss(1989)
2.2 Reiss(1989),Billingsley(1995)
2.7 Johnson(2004)
2.12 LeCam(1960)
3.1 Feller(1966),Ferguson(1996)
3.4 Breiman(1968,1992)
3.5 Breiman(1968,1992)
3.6 Provedinthetext
3.9 BickelandDoksum(2001)
4.1,4.3 Brown,Cai,andDasGupta(2006)
5.1 Billingsley(1995)
5.9 Anscombe(1952)
5.10,5.11 Breiman(1968,1992),Ash(1973)
5.13,5.16,5.18,5.19 Breiman(1968,1992)
6.1,6.2 Billingsley(1995)
6.6 Shiryaev(1980)
7.1,7.2,7.3 Serfling(1980)
7.5 KoenkerandBassett(1978)
8.4,8.5,8.6 Galambos(1977)
8.12,8.13,8.14 Galambos(1977)
8.16,8.17 Leadbetteretal.(1983)
xiii