Large Sample Methods in Statistics: An Introduction with Applications

386 Pages·1993·10.932 MB·English

by  Pranab Kumar Sen, Julio M. Singer (auth.)

LARGE
SAMPLE
METHODS
IN
STATISTICS
AN INTRODUCTION
WITH APPLICATIONS
LARGE
SAMPLE
METHODS
IN
STATISTICS
AN INTRODUCTION
WITH APPLICATIONS
Pranab K. Sen
Julio M. Singer
SPRINGER-SCIENCE+BUSINESS MEDIA, B. V.
© Springer Science+Business Media Dordrecht 1993
Originally published by Chapman & Hall Inc New York in 1993
Softcover reprint of the hardcover 1st edition 1993
All rights reserved. No part of this book may be reprinted or reproduced or utilized
in any form or by any electronic, mechanical or other means, now known or
hereafter invented, including photocopying and recording, or by an information
storage or retrieval system, without permission in writing from the publishers.
Library of Congress Cataloging-in-Publication Data
Sen, Pranab Kumar, 1937-
Large sample methods in statistics : an introduction with applications / by
Pranab K. Sen, Julio M. Singer.
p. cm.
Includes bibliographical references and index.
ISBN 978-0-412-04221-8 ISBN 978-1-4899-4491-7 (eBook)
DOI 10.1007/978-1-4899-4491-7
1. Asymptotic distribution (Probability theory) 2. Stochastic processes.
I. Singer, Julio da Motta, 1950- . II. Title.
QA273.6.S46 1993
519.5'2-dc20 92-46163
CIP
British Library Cataloguing in Publication Data also available.
To our inspiring mothers
Kalyani Sen and Edith Singer
Contents
Preface Xl
1 Objectives and Scope: General Introduction 1
1.1 Introduction 1
1.2 Large sample methods: an overview of applications 3
1.3 The organization of this book 10
1.4 Basic tools and concepts 16
1.5 Concluding notes 29
1.6 Exercises 29
2 Stochastic Convergence 31
2.1 Introduction 31
2.2 Modes of stochastic convergence 34
2.3 Probability inequalities and laws of large numbers 48
2.4 Inequalities and laws of large numbers for some dependent
variables 72
2.5 Some miscellaneous convergence results 85
2.6 Concluding notes 91
2.7 Exercises 92
3 Weak Convergence and Central Limit Theorems 97
3.1 Introduction 97
3.2 Some important tools 102
3.3 Central limit theorems 107
3.4 Projection results and variance-stabilizing transformations 125
3.5 Rates of convergence to normality 147
3.6 Concluding notes 151
3.7 Exercises 152
viii CONTENTS
4 Large Sample Behavior of Empirical Distributions and Or-
der Statistics 155
4.1 Introduction 155
4.2 Preliminary notions 157
4.3 Sample quantiles 166
4.4 Extreme order statistics 173
4.5 Empirical distributions 184
4.6 Functions of order statistics and empirical distributions 188
4.7 Concluding notes 195
4.8 Exercises 195
5 Asymptotic Behavior of Estimators and Test Statistics 201
5.1 Introduction 201
5.2 Asymptotic behavior of maximum likelihood estimators 202
5.3 Asymptotic properties of U-statistics and related estimators 210
5.4 Asymptotic behavior of other classes of estimators 219
5.5 Asymptotic efficiency of estimators 228
5.6 Asymptotic behavior of some test statistics 234
5.7 Concluding notes 244
5.8 Exercises 246
6 Large Sample Theory for Categorical Data Models 247
6.1 Introduction 247
6.2 Nonparametric goodness-of-fit tests 249
6.3 Estimation and goodness-of-fit tests: parametric case 253
6.4 Asymptotic theory for some other important statistics 262
6.5 Concluding notes 266
6.6 Exercises 266
7 Large Sample Theory for Regression Models 273
7.1 Introduction 273
7.2 Generalized least-squares procedures 276
7.3 Robust estimators 291
7.4 Generalized linear models 300
7.5 Generalized least-squares versus generalized estimating
equations 314
7.6 No nparametric regression 317
7.7 Concluding notes 322
7.8 Exercises 323
CONTENTS ix
8 Invariance Principles in Large Sample Theory 327
8.1 Introduction 327
8.2 Weak invariance principles 328
8.3 Weak convergence of partial sum processes 332
8.4 Weak convergence of empirical processes 343
8.5 Weak convergence and statistical functionals 354
8.6 Weak convergence and nonparametrics 360
8.7 Strong invariance principles 366
8.8 Concluding notes 367
8.9 Exercises 368
References 371
Index 377
Preface
Students and investigators working in Statistics, Biostatistics or Applied
Statistics in general are constantly exposed to problems which involve large
quantities of data. Since in such a context, exact statistical inference may
be computationally out ofreach and in many cases not even mathematically
tractable, they have to rely on approximate results. Traditionally, the justi
fication for these approximations was based on the convergence of the first
four moments of the distributions of the statistics under investigation to
those of some normal distribution. Today we know that such an approach is
not always theoretically adequate and that a somewhat more sophisticated
set of techniques based on the convergence of characteristic functions may
provide the appropriate justification. This need for more profound mathe
matical theory in statistical large sample theory is even more evident if we
move to areas involving dependent sequences of observations, like Survival
Analysis or Life Tables; there, some use of martingale structures has dis
tinct advantages. Unfortunately, most of the technical background for the
understanding of such methods is dealt with in specific articles or textbooks
written for an audience with such a high level of mathematical knowledge,
that they exclude a great portion of the potential users.
This book is intended to cover this gap by providing a solid justifica
tion for such asymptotic methods, although at an intermediate level. It
focuses primarily on the basic tools of conventional large sample theory for
independent observations, but also provides some insight to the rationale
underlying the extensions of these methods to more complex situations in
volving dependent measurements. The main thrust is on the basic concepts
of convergence and asymptotic distribution theory for a large class of statis
tics commonly employed in diverse practical problems. Chapter 1 describes
the type of problems considered in the text along with a brief summary
of some basic mathematical and statistical concepts required for a good
understanding of the remaining chapters. Chapters 2 and 3 contain the es
sential tools needed to prove asymptotic results for independent sequences
of random variables as well as an outline of the possible extensions to cover
the dependent sequence case. Chapter 4 explores the relationship between
xii PREFACE
order statistics and empirical distribution functions with respect to their
asymptotic properties and illustrates their use in some applications. Chap
ter 5 discusses some general results on the asymptotics of estimators and
test statistics; their actual application to Categorical Data and Regression
Analysis is illustrated in Chapters 6 and 7, respectively. Finally, Chapter 8
deals with an introductory exposition of the technical background required
to deal with the asymptotic theory for statistical functionals. The objec
tive here is to provide some motivation and the general flavor of the prob
lems in this area, since a rigorous treatment would require a much higher
level of mathematical background, than the one we contemplate. The eight
chapters were initially conceived for a one-semester course for second year
students in Biostatistics or Applied Statistics doctoral programs as well as
for last year undergraduate or first year graduate programs in Statistics. A
more realistic view, however, would restrict the material for such purposes
to the first five chapters along with a glimpse into Chapter 8. Chapters
6 and 7 could be included as supplementary material in Categorical Data
and Linear Models courses, respectively. Since the text includes a number
of practical examples, it may be useful as a reference text for investigators
in many areas requiring the use of Statistics.
The authors would like to thank the numerous students who took Large
Sample Theory courses at the Department of Biostatistics, University of
North Carolina at Chapel Hill and Department of Statistics, University of
Sao Paulo, providing important contributions to the design of this text. We
would also like to thank Ms. Denise Morris, Ms. M6nica Casajus and Mr.
Walter Vicente Fernandes for their patience in the typing of the manuscript.
The editorial assistance provided by Antonio Carlos Lima with respect to
handling 'lEX and UTEX was crucial to the completion of this project. We
are also grateful to Dr. Jose Galvao Leite and Dr. Bahjat Qaqish for their
enlightening comments and careful revision of portions of the manuscript.
Finally we must acknowledge the Cary C. Boshamer Foundation, Univer
sity of North Carolina at Chapel Hill as well as Conselho Nacional de De
senvolvimento Cientifico e Tecnol6gico, Brazil and Fundac;ao de Amparo
a Pesquisa do Estado de Sao Paulo, Brazil for providing financial support
during the years of preparation of the text.