Linear Models and Generalizations: Least Squares and Alternatives

582 Pages·2008·4.18 MB·English

by  Professor C. Radhakrishna Rao

Springer Series in Statistics
Advisors:
P.Bickel,P.Diggle,S.Fienberg,
U.Gather,I.Olkin,S.Zeger
Springer Series in Statistics
Alho/Spencer:StatisticalDemographyandForecasting.
Andersen/Borgan/Gill/Keiding:StatisticalModelsBasedonCountingProcesses.
Arnold/Castillo/Sarabia:ConditionalSpecificationofStatisticalModels.
Atkinson/Riani/Cerioli:ExploringMultivariateDatawiththeForwardSearch.
Atkinson/Riani:RobustDiagnosticRegressionAnalysis.
Berger:StatisticalDecisionTheoryandBayesianAnalysis.
Borg/Groenen:ModernMultidimensionalScaling.
Bremaud:PointProcessesandQueues:MartingaleDynamics.
Brockwell/Davis:TimeSeries:TheoryandMethods.
Bucklew:IntroductiontoRareEventSimulation.
Cappé/Moulines/Ryden:InferenceinHiddenMarkovModels.
Chan/Tong:Chaos:AStatisticalPerspective.
Chen/Shao/Ibrahim:MonteCarloMethodsinBayesianComputation.
Coles:AnIntroductiontoStatisticalModelingofExtremeValues.
David/Edwards:AnnotatedReadingsintheHistoryofStatistics.
Devroye/Lugosi:CombinatorialMethodsinDensityEstimation.
Diggle/Ribeiro:Model-basedGeostatistics.
Dudoit/vanderLaan:MultipleTestingProcedureswithApplicationsto
Genomics.
Dzhaparidze:ParameterEstimationandHypothesisTestinginSpectralAnalysis
ofStationaryTimeSeries.
Efromovich:NonparametricCurveEstimation.
Eggermont/LaRiccia:MaximumPenalizedLikelihoodEstimation·VolumeI.
Fahrmeir/Tutz:MultivariateStatisticalModellingBasedonGeneralizedLinear
Models.
Fan/Yao:NonlinearTimeSeries.
Farebrother:FittingLinearRelationships.
Ferraty/Vieu:NonparametricFunctionalDataAnalysis.
Ferreira/Lee:MultiscaleModeling.
Fienberg/Hoaglin:SelectedPapersofFrederickMosteller.
Frühwirth-Schnatter:FiniteMixtureandMarkovSwitchingModels.
Ghosh/Ramamoorthi:BayesianNonparametrics.
Glaz/Naus/Wallenstein:ScanStatistics.
Good:Permutation,Paramatric,andBootstrapTestsofHypotheses.
Gourieroux:ARCHModelsandFinancialApplications.
Gu:SmoothingSplineANOVAModels.
Györfi/Kohler:ADistribution-FreeTheoryofNonparametricRegression.
Habermann/Krzyzak/Walk:AdvancedStatistics.
Hall:TheBootstrapandEdgeworthExpansion.
Harrell:RegressionModelingStrategies.
Hart:NonparametricSmoothingandLack-of-FitTests.
Hastie/Tibshirani/Friedman:TheElementsofStatisticalLearning.
(continuedafterindex)
C. Radhakrishna Rao
Helge Toutenburg
Shalabh
Christian Heumann
Linear Models
and Generalizations
Least Squares and Alternatives
Third Extended Edition
With Contributions by Michael Schomaker
123
ProfessorC.RadhakrishnaRao Dr.Shalabh
DepartmentofStatistics DepartmentofMathematics
326ThomasBuilding &Statistics
PennsylvaniaStateUniversity IndianInstituteofTechnology
UniversityPark,PA16802 Kanpur−208016
USA India
[email protected] [email protected]
ProfessorHelgeToutenburg Dr.ChristianHeumann
InstitutfürStatistik InstitutfürStatistik
Ludwig-Maximilians-Universität Ludwig-Maximilians-Universität
Akademiestraße1 Akademiestraße1
80799München 80799München
Germany Germany
helge.toutenburg@ [email protected]
stat.uni-muenchen.de
LibraryofCongressControlNumber:2007934936
ISBN978-3-540-74226-5 3rdeditionSpringerBerlinHeidelbergNewYork
ISBN978-0-387-98848-1 2ndeditionSpringerBerlinHeidelbergNewYork
Thisworkissubjecttocopyright.Allrightsarereserved,whetherthewholeorpartofthematerial
is concerned, specificallythe rights of translation, reprinting, reuseof illustrations, recitation,
broadcasting,reproductiononmicrofilmorinanyotherway,andstorageindatabanks.Duplication
ofthispublicationorpartsthereofispermittedonlyundertheprovisionsoftheGermanCopyright
LawofSeptember9,1965,initscurrentversion,andpermissionforusemustalwaysbeobtained
fromSpringer.ViolationsareliabletoprosecutionundertheGermanCopyrightLaw.
SpringerisapartofSpringerScience+BusinessMedia
springer.com
©Springer-VerlagBerlinHeidelberg1995,1999,2008
Theuseofgeneraldescriptivenames,registerednames,trademarks,etc.inthispublicationdoes
notimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Production:LE-TEXJelonek,Schmidt&VöcklerGbR,Leipzig
Cover-design:WMXDesignGmbH,Heidelberg
SPIN12108695 154/3180YL-543210 Printedonacid-freepaper
Preface to the First Edition
Thebookisbasedonseveralyearsofexperienceofbothauthorsinteaching
linearmodelsatvariouslevels.Itgivesanup-to-dateaccountofthe theory
and applications of linear models. The book can be used as a text for
courses in statistics at the graduate level and as an accompanyingtext for
courses in other areas. Some of the highlights in this book are as follows.
A relatively extensive chapter on matrix theory (Appendix A) provides
the necessary tools for proving theorems discussed in the text and offers a
selectionofclassicalandmodernalgebraicresultsthatareusefulinresearch
work in econometrics, engineering, and optimization theory. The matrix
theory of the last ten years has produced a series of fundamental results
aboutthedefinitenessofmatrices,especiallyforthedifferencesofmatrices,
which enable superiority comparisons of two biased estimates to be made
for the first time.
We have attempted to provide a unified theory of inference from linear
models with minimal assumptions. Besides the usual least-squares theory,
alternative methods of estimation and testing based on convex loss func-
tions and general estimating equations are discussed. Special emphasis is
given to sensitivity analysis and model selection.
A special chapter is devoted to the analysis of categoricaldata based on
logit, loglinear, and logistic regressionmodels.
The material covered, theoretical discussion, and a variety of practical
applications will be useful not only to students but also to researchersand
consultants in statistics.
WewouldliketothankourcolleaguesDr.G.TrenklerandDr.V.K.Sri-
vastava for their valuable advice during the preparation of the book. We
vi Preface to the First Edition
wish to acknowledge our appreciation of the generous help received from
AndreaScho¨pp,AndreasFieger,andChristianKastnerforpreparingafair
copy.Finally,wewouldliketothankDr.MartinGilchristofSpringer-Verlag
for his cooperation in drafting and finalizing the book.
We request that readers bring to our attention any errors they may
find in the book and also give suggestions for adding new material and/or
improving the presentation of the existing material.
University Park, PA C. Radhakrishna Rao
Mu¨nchen, Germany Helge Toutenburg
July 1995
Preface to the Second Edition
The first edition of this book has found wide interest in the readership.
A first reprint appeared in 1997 and a special reprint for the Peoples Re-
public of China appeared in 1998. Based on this, the authors followed
the invitation of John Kimmel of Springer-Verlag to prepare a second edi-
tion, which includes additional material such as simultaneous confidence
intervalsfor linear functions,neuralnetworks,restrictedregressionandse-
lectionproblems(Chapter3);mixedeffectmodels,regression-likeequations
in econometrics, simultaneous prediction of actual and average values, si-
multaneousestimationofparametersindifferentlinearmodelsbyempirical
Bayessolutions(Chapter4);themethodoftheKalmanFilter(Chapter6);
and regression diagnostics for removing an observation with animating
graphics (Chapter 7).
Chapter 8, “Analysis of Incomplete Data Sets”, is completely rewrit-
ten, including recent terminology and updated results such as regression
diagnostics to identify Non-MCAR processes.
Chapter 10, “Models for Categorical Response Variables”, also is com-
pletely rewritten to present the theory in a more unified way including
GEE-methods for correlatedresponse.
At the end of the chapters we have given complements and exercises.
We have added a separate chapter (Appendix C) that is devoted to the
software available for the models covered in this book.
We would like to thank our colleagues Dr. V. K. Srivastava (Lucknow,
India)andDr.C.Heumann(Mu¨nchen,Germany)fortheirvaluableadvice
during the preparation of the second edition. We thank Nina Lieske for
her help in preparing a fair copy. We would like to thank John Kimmel of
viii Preface to theSecond Edition
Springer-Verlagforhiseffectivecooperation.Finally,wewishtoappreciate
the immense work done by Andreas Fieger (Mu¨nchen, Germany) with re-
spect to the numericalsolutions of the examples included, to the technical
managementofthecopy,andespeciallytothereorganizationandupdating
of Chapter 8 (including some of his own research results). Appendix C on
software was written by him, also.
We request that readers bring to our attention any suggestions that
would help to improve the presentation.
University Park, PA C. Radhakrishna Rao
Mu¨nchen, Germany Helge Toutenburg
May 1999
Preface to the Third Edition
The authors of the earlier editions of the book - C. Radhakrishna Rao
andHelgeToutenburg-receivedvarioussuggestionsfromreadersafterthe
publicationofthesecondedition.Basedonthewidespreadreadershipand
reviews of the book, they decided to revise the second edition of the book
with two more authors - Shalabh and Christian Heumann. Most of the
chapters are updated with recent developments in the area of linear mod-
els and more topics are included. The relationship between the theory of
linearmodelsandregressionanalysisisdiscussedinchapter1.Chapter2on
simple linear models is newly introduced for a better understanding of the
conceptsinfurtherchapters.Italsodiscusssomealternativeregressionap-
proaches like orthogonalregression,reduced major axis regression,inverse
regression, LAD regression besides the direct regression and the method
of maximum likelihood. Such approaches are generally not discussed in
most of the books on linear models and regressionanalysis.Many parts in
chapter3arerewrittenandupdated.Thetopicsonregressionwithstochas-
tic explanatoryvariables,Stein–rule estimation, nonparametricregression,
classificationandregressiontrees,boostingandbagging,testsofparameter
constancy, balanced loss function and LINEX loss function are important
topics included beside others. Some parts in chapter 4 are rewritten and
topics on linear mixed models with normally distributed errors and ran-
domeffects,andmeasurementerrormodelsareintroduced.The maximum
likelihoodestimationandStein-ruleestimationunderlinearrestrictionsare
addedinchapter5.Thetopicsonpredictionregionsandsimultaneouspre-
diction of actual and average values of the study variable are expanded in
chapter 6. Model selection criteria are introduced in chapter 7. A section
x Preface to theThird Edition
on the treatment of nonignorable nonresponse is introduced in chapter 8.
The full likelihood approach for marginal models for categorical response
variable is expanded and updated in chapter 10.
We owe our thanks to Late Professor V. K. Srivastava (India) for his
continuous encouragement. We thank John Kimmel and Lilith Braun of
Springer-Verlag for their help in the third edition of the book.
We invite the readers to send their comments and suggestions on the
contents and treatment of the topics in the book for possible improvement
in future editions.
University Park, PA C. Radhakrishna Rao
Mu¨nchen, Germany Helge Toutenburg
Kanpur, India Shalabh
Mu¨nchen, Germany Christian Heumann
July 2007