Probability theory and statistical inference: econometric modeling with observational data

842 Pages·1999·5.241 MB·English

by  Aris Spanos

Probability Theory
and Statistical Inference:
Econometric Modeling
with Observational Data
Aris Spanos
CAMBRIDGE UNIVERSITY PRESS
Probability Theory and Statistical Inference
This major new textbook from a distinguished econometrician is intended for
students taking introductory courses in probability theory and statistical infer-
ence.No prior knowledge other than a basic familiarity with descriptive statis-
tics is assumed.
The primary objective of this book is to establish the framework for the
empirical modeling ofobservational (non-experimental) data.This framework
is formulated with a view to accommodating the peculiarities of observational
(as opposed to experimental) data in a unifying and logically coherent way.
Probability Theory and Statistical InferencediVers from traditional textbooks
in so far as it emphasizes concepts, ideas, notions, and procedures which are
appropriate for modeling observational data. Special emphasis is placed on
relating probabilistic concepts to chance regularity patterns exhibited by
observed data.
Aimed primarily at students at second-year undergraduate level and above
studying econometrics and economics,this textbook will also be useful for stu-
dents in other disciplines which make extensive use of observational data,
including finance,biology,sociology,education,psychology,and climatology.
aris spanos is a leading researcher and teacher in econometrics.He is cur-
rently based at the University ofCyprus and Virginia Polytechnic Institute and
State University and has taught previously at institutions including,Birkbeck
College, the University of Cambridge, and the University of California. His
previous books include Statistical Foundations ofEconometric Modelling,pub-
lished by Cambridge University Press in 1986.
Probability Theory
and Statistical Inference
Econometric Modeling with
Observational Data
Aris Spanos
PUBLISHED BY CAMBRIDGE UNIVERSITY PRESS (VIRTUAL PUBLISHING)
FOR AND ON BEHALF OF THE PRESS SYNDICATE OF THE UNIVERSITY OF CAMBRIDGE
The Pitt Building, Trumpington Street, Cambridge CB2 IRP
40 West 20th Street, New York, NY 10011-4211, USA
477 Williamstown Road, Port Melbourne, VIC 3207, Australia

http://www.cambridge.org

© Cambridge University Press 1999
This edition © Cambridge University Press (Virtual Publishing) 2003

First published in printed format 1999

A catalogue record for the original printed book is available
from the British Library and from the Library of Congress
Original ISBN 0 521 41354 0 hardback
Original ISBN 0 521 42408 9 paperback

ISBN 0 511 01097 4 virtual (netLibrary Edition)
Contents
Preface pagexi
Acknowledgments xxiv
1 An introduction to empirical modeling 1
1.1 Introduction 1
1.2 Stochastic phenomena,a preliminary view 3
1.3 Chance regularity and statistical models 13
1.4 Statistical adequacy 16
1.5 Statistical versus theory information* 19
1.6 Observed data 20
1.7 Looking ahead 29
1.8 Exercises 30
2 Probability theory:a modeling framework 31
2.1 Introduction 31
2.2 Simple statistical model:a preliminary view 33
2.3 Probability theory:an introduction 39
2.4 Random experiments 42
2.5 Formalizing condition [a]:the outcomes set 45
2.6 Formalizing condition [b]:events and probabilities 48
2.7 Formalizing condition [c]:random trials 69
2.8 Statistical space 73
2.9 A look forward 74
2.10 Exercises 75
3 The notion ofa probability model 77
3.1 Introduction 77
3.2 The notion ofa simple random variable 78
3.3 The general notion ofa random variable 85
3.4 The cumulative distribution and density functions 89
v
vi Contents
3.5 From a probability space to a probability model 97
3.6 Parameters and moments 104
3.7 Moments 109
3.8 Inequalities 131
3.9 Summary 132
3.10 Exercises 133
Appendix A Univariate probability models 135
A.1 Discrete univariate distributions 136
A.2 Continuous univariate distributions 138
4 The notion ofa random sample 145
4.1 Introduction 145
4.2 Joint distributions 147
4.3 Marginal distributions 155
4.4 Conditional distributions 158
4.5 Independence 167
4.6 Identical distributions 171
4.7 A simple statistical model in empirical modeling:a preliminary view 175
4.8 Ordered random samples* 181
4.9 Summary 184
4.10 Exercises 184
Appendix B Bivariate distributions 185
B.1 Discrete bivariate distributions 185
B.2 Continuous bivariate distributions 186
5 Probabilistic concepts and real data 190
5.1 Introduction 190
5.2 Early developments 193
5.3 Graphic displays:a t-plot 195
5.4 Assessing distribution assumptions 197
5.5 Independence and the t-plot 212
5.6 Homogeneity and the t-plot 217
5.7 The empirical cdfand related graphs* 229
5.8 Generating pseudo-random numbers* 254
5.9 Summary 258
5.10 Exercises 259
6 The notion ofa non-random sample 260
6.1 Introduction 260
6.2 Non-random sample:a preliminary view 263
6.3 Dependence between two random variables:joint distributions 269
6.4 Dependence between two random variables:moments 272
6.5 Dependence and the measurement system 282
6.6 Joint distributions and dependence 290
Contents vii
6.7 From probabilistic concepts to observed data 309
6.8 What comes next? 330
6.9 Exercises 335
7 Regression and related notions 337
7.1 Introduction 337
7.2 Conditioning and regression 339
7.3 Reduction and stochastic conditioning 356
7.4 Weak exogeneity* 366
7.5 The notion ofa statistical generating mechanism (GM) 368
7.6 The biometric tradition in statistics 377
7.7 Summary 397
7.8 Exercises 397
8 Stochastic processes 400
8.1 Introduction 400
8.2 The notion ofa stochastic process 403
8.3 Stochastic processes:a preliminary view 410
8.4 Dependence restrictions 420
8.5 Homogeneity restrictions 426
8.6 “Building block”stochastic processes 431
8.7 Markov processes 433
8.8 Random walk processes 435
8.9 Martingale processes 438
8.10 Gaussian processes 444
8.11 Point processes 458
8.12 Exercises 460
9 Limit theorems 462
9.1 Introduction to limit theorems 462
9.2 Tracing the roots oflimit theorems 465
9.3 The Weak Law ofLarge Numbers 469
9.4 The Strong Law ofLarge Numbers 476
9.5 The Law ofIterated Logarithm* 481
9.6 The Central Limit Theorem 482
9.7 Extending the limit theorems* 491
9.8 Functional Central Limit Theorem* 495
9.9 Modes ofconvergence 503
9.10 Summary and conclusion 510
9.11 Exercises 510
10 From probability theory to statistical inference* 512
10.1 Introduction 512
10.2 Interpretations ofprobability 514
viii Contents
10.3 Attempts to build a bridge between probability and observed data 520
10.4 Toward a tentative bridge 528
10.5 The probabilistic reduction approach to specification 541
10.6 Parametric versus non-parametric models 546
10.7 Summary and conclusions 556
10.8 Exercises 556
11 An introduction to statistical inference 558
11.1 Introduction 558
11.2 An introduction to the classical approach 559
11.3 The classical versus the Bayesian approach 568
11.4 Experimental versus observational data 570
11.5 Neglected facets ofstatistical inference 575
11.6 Sampling distributions 578
11.7 Functions ofrandom variables 584
11.8 Computer intensive techniques for approximating sampling
distributions* 594
11.9 Exercises 600
12 Estimation I:Properties ofestimators 602
12.1 Introduction 602
12.2 Defining an estimator 603
12.3 Finite sample properties 607
12.4 Asymptotic properties 615
12.5 The simple Normal model 621
12.6 SuYcient statistics and optimal estimators* 627
12.7 What comes next? 635
12.8 Exercises 635
13 Estimation II:Methods ofestimation 637
13.1 Introduction 637
13.2 Moment matching principle 639
13.3 The least-squares method 648
13.4 The method ofmoments 654
13.5 The maximum likelihood method 659
13.6 Exercises 678
14 Hypothesis testing 681
14.1 Introduction 681
14.2 Leading up to the Fisher approach 682
14.3 The Neyman–Pearson framework 692
14.4 Asymptotic test procedures* 713
14.5 Fisher versus Neyman–Pearson 720
Contents ix
14.6 Conclusion 727
14.7 Exercises 727
15 Misspecification testing 729
15.1 Introduction 729
15.2 Misspecification testing:formulating the problem 733
15.3 A smorgasbord ofmisspecification tests 739
15.4 The probabilistic reduction approach and misspecification 753
15.5 Empirical examples 765
15.6 Conclusion 783
15.7 Exercises 784
References 787
Index 806
Preface
1 Intended audience and distinguishing features
This is a textbook intended for an introductory course in probability theoryand statisti-
cal inference,written for students who have had at least a semester course in calculus.The
additional mathematics needed are coalesced into the discussion to make it self-
contained,paying particular attention to the intuitive understanding of the mathemat-
ical concepts.No prerequisites in probability and statistical inference are required but some
familiarity with descriptive statistics will be ofvalue.
The primary objective of this book is to lay the foundations and assemble the over-
arching framework for the empirical modeling ofobservational(non-experimental) data.
This framework,known as probabilistic reduction,is formulated with a view to accom-
modating the peculiarities ofobservational (as opposed to experimental) data in a unify-
ing and logically coherent way. It diVers from traditional textbooks in so far as it
emphasizes concepts,ideas,notions,and procedures which are appropriate for modeling
observational data.
The primary intended audience of this book includes interested undergraduate and
graduate students of econometrics as well as practicing econometricians who have been
trained in the traditional textbook approach.Special consideration has been given to the
needs ofthose using the textbook for self-study.This text can also be used by students of
other disciplines, such as biology, sociology, education, psychology, and climatology,
where the analysis ofobservational data is ofinterest.
The traditional statistical literature over the last 50 years or so,has focused,almost
exclusively, on procedures and methods appropriate for the analysis of experimental-
type (experimental and sample survey) data.The traditional modeling framework has
been that of the experimental designtradition,as molded by Fisher (1935) (and formu-
lated by the sample survey literature ofthe 1940s and 1950s),and the “curve fitting”per-
spective of the least-squares tradition. Both of these traditions presume a modeling
framework in the context of which the observed data are interpreted as a realization of
an observable phenomenon which can be realistically viewed as a nearly isolated (by
divine or human intervention) system; see Spanos (1995b). This book purports to
redress (somewhat) the balance by expounding a modeling framework appropriate for
observational data. This modeling framework can be viewed in the spirit of the
xi