Estimation and Inference in Econometrics

903 Pages·2021·9.533 MB·English

by  Russell Davidson, James G. MacKinnon

Estimation and Inference
in Econometrics
by
Russell Davidson
and
James G. MacKinnon
Online version of September, 2021
Oxford University Press, New York
ISBN 0-19-506011-3
This book was originally published by Oxford University Press
in late 1992 with a 1993 date. By agreement with OUP,
the rights reverted to the authors in 2021.
Copyright (cid:13)c 1993 by Oxford University Press, Inc.
Copyright (cid:13)c 2021 by Russell Davidson and James G. MacKinnon
ISBN 0-19-506011-3, HB139.D368 1993
To our students
Preface
When we began writing this book, longer ago than we care to admit, our goal
was to write a text that could be used for the second and third semesters
of a typical graduate sequence in econometrics. We perceived a lack of any
textbook that addressed the needs of students trying to acquire a solid un-
derstanding of what we think of as the “modern” approach to econometrics.
By this we mean an approach that goes beyond the well-known linear re-
gression model, stresses the essential similarities of all the leading estimation
methods, and puts as much emphasis on testing and model specification as
on estimation.
We soon realized that this plan had a fatal flaw. In order to write a book
for the second course in econometrics, one must be confident of what will have
been covered in the first course. Since there was not then, and is not now,
a widely accepted standard syllabus for the first course in econometrics, we
decided that we would have to start there. We therefore changed our plan,
and this book now attempts to develop econometric theory from the ground
up. Readers are of course expected to have some acquaintance with element-
ary econometrics before starting, but no more than would be part of a typical
undergraduate curriculum. They are also expected to have the mathematical
maturity appropriate to graduate students in economics, although we do pro-
vide two appendices that cover the mathematical and statistical prerequisites
for understanding the material.
Almost all of the econometric theory we present is asymptotic, which
means that it is exactly true only in the limit as the sample size tends to
infinity, but is thought (or rather hoped) to be approximately true in finite
samples. In recent years, researchers have found it increasingly necessary to
go beyond the confines of the standard linear regression model, in which re-
strictive classical assumptions lead to exact results about the distributions
of the ordinary least squares estimator and of the familiar t and F statistics.
Greatergeneralityinmodelspecification, however, carriesthepricethatexact
finite-sample results are very rarely available. Happily, asymptotic economet-
ric theory is now at a mature stage of development, and it provides the main
theoretical foundation for the present book.
Our first chapter does not really discuss econometrics at all. Instead,
it presents those aspects of the geometry of least squares that are needed in
the rest of the book. A key result in this context is the theorem that we
have dubbed the Frisch-Waugh-Lovell Theorem. We have found that get-
ting students to understand this theorem, although often a rather challenging
task, does a great deal to develop intuition about estimation and testing in
econometrics. A particular application of the theorem, which we present in
viii Preface
Chapter 1, is to the question of leverage and influence in regression models.
Existing treatments of this important topic have typically been algebraically
difficult and unintuitive. Use of the FWL Theorem makes it possible to de-
velop a much simpler treatment. Chapter 1 also briefly discusses the compu-
tation of ordinary least squares estimates, a subject about which too many
students of econometrics are completely ignorant.
One of our aims in this book is to emphasize nonlinear estimation. In
Chapters 2 and 3, we therefore plunge directly into a treatment of the non-
linear regression model. It turns out that it is scarcely any more difficult to
develop the essential notions of least-squares estimation, and of statistical in-
ference based on such estimation, in a nonlinear context than it is in the more
usual linear one. In fact, the essential notions are often easier to come to
grips with if one is not distracted by the great wealth of detailed but special
results that enrich the linear theory.
After the largely intuitive treatment of Chapters 2 and 3, we provide
in Chapters 4 and 5 a fuller and more rigorous account of the asymptotic
theory that underlies the nonlinear regression model. Just how far to go
in the quest for rigor has been a thorny problem at many points. Much of
the recent literature in theoretical econometrics appears to be inaccessible to
many students, in large part, we believe, because rigor has taken precedence
over the communication of fundamental ideas. We have therefore deliberately
not aimed at the same standards of rigor. On the other hand, some rigor is
needed in any account that is not merely anecdotal. It is in Chapters 4 and 5,
and later in Chapter 8, which lays the foundations of the theory of maximum
likelihood, that we have gone as far as we felt we could in the direction of a
formalrigoroustreatment. Attimesweevenadopta“theorem-proof”format,
something that we have generally avoided in the book. Many instructors will
prefer to skim these chapters, especially in a first course, although we hope
that most will choose not to omit them entirely.
Although we stress nonlinear models throughout the book, we also em-
phasize another point that has emerged in the last fifteen years and that has
been a central focus of much of our own research over that period. In order
to perform statistical inference on the results of a nonlinear estimation pro-
cedure, it is almost always possible to make use of artificial linear regressions
for the purposes of computing test statistics. Chapter 6 is the first chapter
in which we discuss an artificial linear regression, and it is a key chapter for
understanding much subsequent material. We show how the so-called Gauss-
Newton regression can be used for a variety of purposes, most notably the
calculation of Lagrange multiplier tests and related test statistics, the compu-
tation of nonlinear least squares estimates, and the computation of one-step
efficientestimates. Theuseofartificialregressionsfordoingdiagnostictestsof
model specification is emphasized. Other artificial regressions are introduced
later in the book for use in contexts more general than that of nonlinear
regression models, but the intuition is always the same.
Preface ix
OurtreatmentofthelinearsimultaneousequationsmodelbeginsinChap-
ter 7, where we discuss single-equation instrumental variables estimation. In
line with our emphasis on nonlinear models, we do not stick with linear in-
strumental variables models only, but also treat the estimation of nonlinear
models by instrumental variables and show how the Gauss-Newton regression
generalizes to such models. We also introduce the important idea of tests of
overidentifying restrictions. However, we do not attempt a full treatment of
the linear simultaneous equations model at this point. We have deliberately
left this topic, often thought of as the centerpiece of econometric theory, until
very late in the book. It is our feeling that modern theory and practice are
driftingawayfromthelinearsimultaneousequationsmodel, infavorofamore
flexible approach in which instrumental variables continue to play a large role
but in a much more general context.
The presentation of standard maximum likelihood theory in Chapter 8
reliesasmuchaspossibleoninsightsdevelopedearlierforthenonlinearregres-
sion model. The basic concepts of consistency and asymptotic normality are
already available and can therefore be dealt with quite swiftly. New concepts
arise in connection with the information matrix equality and the Cram´er-Rao
lower bound. In Chapter 9, maximum likelihood methods find their first ma-
jor application as we develop the methods of generalized least squares. These
methods lead naturally to a discussion of multivariate, but not simultaneous,
models. We also devote a section of this chapter to problems particular to the
analysis of panel data.
Chapter 10 deals with a topic of great concern to all econometricians who
workwithtimeseries: serialcorrelation. Fewtopicsineconometricshavebeen
the subject of so vast a literature, much of which is now somewhat outdated.
Although we make no attempt to give a complete account of this literature,
this chapter is nevertheless one of the longest. It provides a first treatment
of time-series methods, since it is here that we describe autoregressive and
moving average processes. Methods of testing for the presence of these pro-
cesses in the error terms of regression equations, and performing estimation
in their presence, are discussed. Again, we highlight the possibility of us-
ing artificial linear regressions for these purposes. One section is devoted to
the important, and in many texts surprisingly neglected, subject of common
factor restrictions.
Hypothesistestinganddiagnostictesting, alwaysaprimaryconcern, take
center stage again in Chapter 11, which discusses tests based on the Gauss-
Newton regression. Nonnested hypothesis testing is discussed here, and the
principle of Durbin-Wu-Hausman tests, introduced earlier in Chapter 7, is
taken up more fully. In addition, a heteroskedasticity-robust version of the
Gauss-Newton regression is developed, providing a first look at issues that
will be taken up in much more detail in Chapters 16 and 17.
Chapter 12 contains material not found in any other textbook treatment,
to our knowledge. Here, in the simple context of the regression model, we
x Preface
discuss the determinants of test power. We show how tests often have power
to reject false hypotheses or ill-specified models even when the alternative
hypothesis underlying the test is also wrongly specified. The unifying concept
is that of a drifting DGP, a generalization of the Pitman drift of standard
statistical analysis. This concept makes it possible to develop an asymptotic
theory of test power, based on asymptotic noncentrality parameters. The
asymptotic power of a test is shown to depend on just two things: its non-
centrality parameter and its number of degrees of freedom. We also devote a
section to the inverse power function, which has recently been proposed as a
useful and powerful tool for the interpretation of test results. We suspect that
some instructors will choose to skip this chapter, but we feel strongly that any
student who aims to be a specialist in econometrics should be familiar with
this material.
In Chapter 13, we turn again to maximum likelihood estimation and
develop, rather formally, the theory of the classical hypothesis tests, relying
for intuition on some of the material of the preceding two chapters. We treat
not only the well-known trio of the likelihood ratio, Lagrange multiplier, and
Wald tests, but also the C(α) test of Neyman, which is now emerging from
some decades of neglect. The latter test turns out to be particularly easy to
implement by means of artificial regressions. It is in this chapter that the
well-known OPG regression is introduced.
From Chapter 14 until the end of the book, most chapters constitute
relatively self-contained units. In these chapters, we try to discuss many of
the topics of importance in modern econometrics. It is here that some readers
may well feel that we have been hopelessly misguided in our selection and
have left out the one thing that all econometricians must know. In a field as
rapidly growing as econometrics is at the moment, they may well be right. We
have been guided largely by our own interests and tastes, which are inevitably
fallible. Two topics that we could well have discussed if space had permitted
arenonparametricandsemiparametrictechniquesandBayesianmethods. We
apologize to specialists in these fields, offering only the lame excuse that we
are not ourselves specialists in them, and would no doubt have failed to do
them justice.
Chapters 14 and 15 deal respectively with models involving transforma-
tions of the dependent variable and models involving qualitative and limited
dependent variables. Both chapters rely heavily on the theory of estimation
and testing for models estimated by maximum likelihood. Courses with an
applied orientation might want to emphasize these chapters, and theoretical
courses might omit them entirely in favor of more advanced topics.
Chapter 16 deals with a variety of topics, including heteroskedasticity,
skewness and kurtosis, conditional moment tests, and information matrix
tests. Many relatively recent developments are discussed in this chapter,
whichleadsnaturallytoChapter17,onthegeneralizedmethodofmoments,or
GMM. This important estimation technique has not, to our knowledge, been
Preface xi
discussed in any detail in previous textbooks. Our treatment depends heavily
on earlier results for instrumental variables and generalized least squares. It
contains both general results for models estimated by means of any set of
moment conditions, and specific results for linear regression models. For the
latter, we present estimators that are more efficient than ordinary and two-
stage least squares in the presence of heteroskedasticity of unknown form.
A full treatment of the linear simultaneous equations model does not
occur until Chapter 18. One advantage of leaving it until late in the book is
that previous results on instrumental variables, maximum likelihood, and the
generalized method of moments are then available. Thus, in Chapter 18, we
areabletoprovidereasonablyadvanceddiscussionsofLIML,FIML,and3SLS
estimation as applications of general techniques that students have already
learned. The GMM framework also allows us to introduce a variant of 3SLS
that is efficient in the presence of heteroskedasticity of unknown form.
Chapters 19 and 20 complete our discussion of time-series issues. The
first deals with a number of topics that are important for applied work, in-
cluding spurious regressions, dynamic models, and seasonality. The second
deals with two related topics of substantial current interest that have not to
our knowledge been treated in previous textbooks, namely, unit roots and
cointegration. These chapters could be covered immediately after Chapter 10
in a course oriented toward applications, although they do make use of results
from some intervening chapters.
Finally, Chapter 21 provides a reasonably detailed introduction to Monte
Carlo methods in econometrics. These methods are already widely used, and
we believe that their use will increase greatly over the next few years as
computers become cheaper and more powerful.
One possible way in which this book can be used is to start at the be-
ginning and continue until the end. If three semesters are available, such an
approach is not only possible but desirable. If less time is available, however,
there are many possible options. One alternative would be to go only as far
as Chapter 13 and then, if time remains, select a few chapters or topics from
the remainder of the book. Depending on the focus of the course, it is also
possible to skip some earlier chapters, such as Chapters 10 and 12, along with
parts of Chapters 9, 11, and 13.
In some courses, it may be preferable to skip much of the theoretical
material entirely and concentrate on the techniques for estimation and in-
ference, without the underlying theory. In that event, we would recommend
that Chapters 4, 5, and 8 be covered lightly, and that Chapter 13 be skipped
entirely. For Chapter 4, the notions of consistency and asymptotic normality
would need to be treated at some level, but it is possible to be content with
simple definitions. A good deal of conceptual material without much math-
ematical formalism can be found in Section 4.4, in which the key idea of a
data-generating process is defined and discussed. For Chapter 5, the results
on the consistency and asymptotic normality of the nonlinear least squares
xii Preface
estimator should be stated and discussed but need not be proved. The Gauss-
Markov Theorem could also be discussed. In Chapter 8, the first two sections
contain the material necessary for later chapters and are not at all formal
in content. The next six sections could then be skipped. Section 8.9, on
testing, could serve as a simpler replacement for the whole of Chapter 13.
Finally, Section 8.10 forges the link between maximum likelihood theory and
the previously covered material on the nonlinear regression model.
One of us teaches in France, where for several years he has used material
from this book as the basis for a series of courses at the upper undergraduate
and master’s levels. The students have already taken basic courses in math-
ematics and statistics when they enter the program. In the first year, they
are presented with material from the first three chapters and a brief discus-
sion of the main issues of Chapters 4 and 5, followed by Chapters 6 and 7,
and accompanied by problem sets to be worked out on the computer. The
second year embarks on maximum likelihood theory from Chapters 8 and 9,
skips most of Chapter 10 (although the model with AR(1) errors is used as an
important example of the uses of the Gauss-Newton regression), and takes up
the testing material of Chapters 11, 12, and 13, with relatively little emphasis
placed on the last of these. Numerous problem sets accompany the material
of these chapters. The third-year course, which is shorter and is joined by
students from other programs, varies more in content, although Chapter 13 is
always used as a focus for presentation and revision of maximum likelihood
methods and testing procedures. Recently, in fact, the first chapter to be
discussed was the last, Chapter 21, on Monte Carlo methods.
It is our hope that this book will be useful, not only to students, but also
to established researchers in econometrics as a work of reference. Many of
the techniques we describe, especially those based on artificial regressions, are
difficult to find in the literature or can be found only in exceedingly technical
articles. We would especially like to draw attention to Chapter 12, in which
we discuss the determinants of test power and the correct interpretation of
test statistics; Chapter 17, which is one of very few textbook treatments of
the generalized method of moments; and Chapter 21, on Monte Carlo experi-
ments. Inthesechapters, wethinkthatthebookmakesauniquecontribution.
Much of the material in the rest of the book, notably Chapters 6, 11, 16, and
20, is also not to be found in other texts. Even when the material we cover is
relatively familiar, we believe that our way of treating it is often novel enough
to be enlightening.
One advantage of a book over the research literature is that a coher-
ent approach and, perhaps of even greater importance, a coherent notation
can be developed. Thus readers can more readily perceive the relations and
similarities between seemingly disparate techniques and arguments. We will
not pretend either that our notation is always absolutely consistent or that
it was easy to make it even as consistent as it is. For example, the study of
time series has for a long time generated a literature distinctly separate from
Preface xiii
the mainstream of econometrics, and within this literature notational habits
have evolved that are incompatible with those that most econometricians are
used to. Many people, however, would be taken aback if time series results
were presented in a notation too markedly different from that used in the
time series literature. We have tried very hard to use notation that is at once
consistent and intuitive. The reader will be the judge of the extent to which
we have succeeded.
It is inconceivable that a book as long and technical as this one should
be free from errors. All the corrections incorporated in this printing and ones
discovered later are available in electronic form via the Internet; see page 875.
There would have been far more errors if we had not had the help of a great
many people in reading preliminary drafts. They pointed out a disconcert-
ingly large number of mistakes, most merely typographical, but some quite
serious. We are indebted to our students, in both Canada and France, in this
respect. We thank especially Dirk Eddelbu¨ttel, Niels Hansen, Doug Tattrie,
Colin Telmer, and John Touchie for the many hours they devoted to going
through chapter after chapter with a fine-tooth comb. Many of our colleagues
have made extremely valuable suggestions to us. Some suggested topics that
we might otherwise have left out, and others were good enough to provide us
with detailed comments on our preliminary efforts. Our thanks go to Richard
Blundell, Colin Cameron, Gordon Fisher, John Galbraith, Bill Greene, Al-
lan Gregory, Mark Kamstra, Peter Sephton, Gregor Smith, Thanasis Stengos,
Timo Ter¨asvirta, and Diana Whistler. We are also indebted to an anony-
mous reader, who urged us to refocus the book when our original plan proved
infeasible.
It is customary for authors to thank their secretaries for unflagging sup-
port, both technical and moral, in the preparation of their manuscript. This
custom imposes on us the pleasant duty of thanking each other, since the
manuscript was prepared, in TEX, by our own unaided efforts. At times, it
seemed that the intricacies of this peerless computer program would take us
more time to master than the whole of econometrics itself. We owe a debt
of gratitude to Donald Knuth, the original author of TEX, and to the many
other people who have contributed to its development.
Finally, we must give thanks where it is due for a great deal of moral
support, and for much more besides, during the long period when we talked
book, more book, and yet more book. It is with much gratitude that we
record our thanks to our wives, Pamela and Susan.