Regression: Models, Methods and Applications

713 Pages·2013·15.682 MB·English

by  Fahrmeir, L., Kneib, Th., Lang, S., Marx, B.

Regression

Ludwig Fahrmeir (cid:2) Thomas Kneib (cid:2) StefanLang (cid:2)
Brian Marx
Regression
Models, Methods and Applications
123
LudwigFahrmeir StefanLang
DepartmentofStatistics DepartmentofStatistics
UniversityofMunich UniversityofInnsbruck
Munich Innsbruck
Germany Austria
ThomasKneib BrianMarx
ChairofStatistics ExperimentalStatistics
UniversityofGo¨ttingen LouisianaStateUniversity
Go¨ttingen BatonRouge,LA
Germany USA
ISBN978-3-642-34332-2 ISBN978-3-642-34333-9(eBook)
DOI10.1007/978-3-642-34333-9
SpringerHeidelbergNewYorkDordrechtLondon
LibraryofCongressControlNumber:2013934096
(cid:2)c Springer-VerlagBerlinHeidelberg2013
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.Exemptedfromthislegalreservationarebriefexcerptsinconnection
with reviews or scholarly analysis or material supplied specifically for the purpose of being entered
and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of
this publication or parts thereof is permitted only under the provisions of the Copyright Law of the
Publisher’slocation,initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.
PermissionsforusemaybeobtainedthroughRightsLinkattheCopyrightClearanceCenter.Violations
areliabletoprosecutionundertherespectiveCopyrightLaw.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
While the advice and information in this book are believed to be true and accurate at the date of
publication,neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityfor
anyerrorsoromissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,with
respecttothematerialcontainedherein.
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
Preface
Regression is the most popular and commonly used statistical methodology for
analyzing empirical problems in social sciences, economics, and life sciences.
Correspondingly,thereexistalargevarietyofmodelsandinferentialtools,ranging
from conventional linear models to modern non- and semiparametric regression.
Currently available textbooks mostly focus on particular classes of regression
models, however, strongly varying in style, mathematical level, and orientation
towards theory or application. Why then another book on regression? Several
introductorytextbooksareavailableforstudentsandpractitionersindiversefields
ofapplications,buttheydealalmostexclusivelywithlinearregression.Ontheother
hand,mosttextsconcentratingonmodernnon-andsemiparametricmethodsprimar-
ilyaddressreaderswithstrongtheoreticalinterestandmethodologicalbackground,
presupposinga correspondinglyhigh-levelmathematicalbasis. Theyare therefore
lessaccessibletoreadersfromappliedfieldswhoneedtoemploythesemethods.
Theaimofthisbookisanappliedandunifiedintroductionintoparametric,non-,
andsemiparametricregressionthatclosesthegapbetweentheoryandapplication.
The most important models and methods in regression are presented on a solid
formal basis, and their appropriate application is shown through many real data
examplesandcasestudies.Availabilityof(user-friendly)softwarehasbeenamajor
criterion for the methods selected and presented. In our view, the interplay and
balance between theory and application are essential for progress in substantive
disciplines, as well as for the development of statistical methodology, motivated
andstimulatedthroughnewchallengesarisingfrommultidisciplinarycollaboration.
A similar goal, but with somewhat different focus, has been pursued in the book
SemiparametricRegressionbyRuppert,Wand,andCarroll(2003).
Thus,ourbookprimarilytargetsanaudiencethatincludesstudents,teachers,and
practitionersinsocial,economic,andlifesciences,aswellasstudentsandteachers
instatisticsprogramsandmathematiciansandcomputerscientistswithinterestsin
statisticalmodelinganddataanalysis.Itiswrittenatanintermediatemathematical
level and assumes only knowledge of basic probability, calculus, and statistics.
Short parts in the text dealing with more complex details or providing additional
informationstartwiththesymbol andendwith .Thesepartsmaybeomitted
in a first reading without loss of continuity. The most important definitions and
v
vi Preface
statements are concisely summarized in boxes. Two appendices describe required
matrixalgebra,aswellaselementsofprobabilitycalculusandstatisticalinference.
Dependingontheparticularinterests,partsofthebookcanbereadindependently
ofremainingpartsoralsoinmodifiedorder:
(cid:129) Chapter2providesanintroductoryoverviewonparametric,non-,andsemipara-
metric regressionmodels,deliberatelyomittingtechnicaldetailsandinferential
tools.
(cid:129) Chapters1–4canbereadasanintroductiontolinearmodels.
(cid:129) Linear mixed models (Sects.7.1–7.4) can be studied immediately after
Chaps.1–4andbeforereadingChaps.5and 6.
(cid:129) Sections 10.1 and 10.2 on linear quantile regression can be read immediately
afterChaps.1–4.
(cid:129) Chapters1–4,Sects.7.1–7.4,andChaps.8–10canbereadasanintroductionto
parametric and semiparametric regression for continuous responses (including
semiparametricquantileregression).
(cid:129) Chapters1–6compriseparametricregressionmodelsforcontinuousanddiscrete
responses.
Anoverviewofpossiblereadingalternativesisgiveninthefollowingtable(chapters
inbrackets[:::]couldbeomitted):
Description Chapters
Linearmodels 1,2,3,4,[Sects.10.1,10.2]
Linearmixedmodels 1,2,3,4,Sects.7.1–7.4,[Sects.10.1–10.2]
Variableselectioninlinearmodels 3,Sects.4.2–4.4
Generalizedlinearmodels 1,2,3,4,5,[6]
Generalizedlinearmixedmodels 1,2,3,4,5,[6],7
Semiparametricregressionforcontinuous 1,2,3,4,8(excl.pages481ff.),9(excl.
responses(excludingmixedmodels) Sects.9.4,9.6.2),[10]
Semiparametricregressionforcontinuous 1,2,3,4,Sects.7.1–7.4,8,9,[10]
responses(includingmixedmodels)
Manyexamplesandapplicationsfromdiversefieldsillustratemodelsandmeth-
ods.Mostof thedata sets are availablevia theurlhttp://www.regressionbook.org/
andthesymbol addedtoanexampleindicatestheavailabilityofcorresponding
software code from the web site. This facilitates independent work and studies
through real data applications and small case studies. In addition, the web site
providesinformationaboutstatisticalsoftwareforregression.
Highlightsofthebookinclude:
(cid:129) An introductionof regression modelsfromfirst principles, i.e., a complete and
comprehensiveintroductiontothelinearmodelinChaps.3,4,and10
(cid:129) Acoverageoftheentirerangeofregressionmodelsstartingwithlinearmodels,
coveringgeneralizedlinear and mixedmodelsand also including(generalized)
additivemodelsandquantileregression
Preface vii
(cid:129) ApresentationofbothfrequentistandBayesianapproachestoregression
(cid:129) Theinclusionofalargenumberofworkedoutexamplesandcasestudies
(cid:129) Althoughthebookiswrittenintextbookstyl esuitableforstudents,thematerial
isclosetocurrentresearchonadvancedregressionanalysis
ThisbookispartlybasedonaprecedingGermanversionthathasbeentranslated
and considerably extended. We are indebted to Alexandra Reuber for translating
largepartsoftheGermanversion.WealsothankHerwigFriedl,ChristianHeumann,
Torsten Hothorn, and Helga Wagner for acting as referees of the book. They
all did a great job and were very helpful in improving the manuscript. Many
thanks to Jesus Crespo Cuaresma, Kathrin Dallmeier, Martin Feldkircher, Oliver
Joost, Franziska Kohl, Jana Lehmann,LorenzOberhammer,Cornelia Oberhauser,
AlexanderRazen, Helene Roth, Judith Santer, Sylvia Schmidt, Nora Seiwald, Iris
Burger,Sven Steinert,NikolausUmlauf,Janette Walde, Elisabeth Waldmann,and
Peter Wechselbergerforsupportandassistance in variousways.Lastbutnotleast
wethankAliceBlanck,AlphonserajaSagayaraj,UlrikeStricker-KombaandNiels
PeterThomasfromSpringerVerlagfortheircontinuedsupportandpatienceduring
thepreparationofthemanuscript.
Mu¨nchen,Germany LudwigFahrmeir
Go¨ttingen,Germany ThomasKneib
Innsbruck,Austria StefanLang
BatonRouge,LA BrianMarx
January2013

Table of Contents
1 Introduction................................................................. 1
1.1 ExamplesofApplications............................................ 4
1.2 FirstSteps............................................................. 11
1.2.1 UnivariateDistributionsoftheVariables.................. 11
1.2.2 GraphicalAssociationAnalysis............................ 13
1.3 NotationalRemarks.................................................. 19
2 RegressionModels.......................................................... 21
2.1 Introduction........................................................... 21
2.2 LinearRegressionModels........................................... 22
2.2.1 SimpleLinearRegressionModel ......................... 22
2.2.2 MultipleLinearRegression ................................ 26
2.3 RegressionwithBinaryResponseVariables:TheLogitModel.... 33
2.4 MixedModels........................................................ 38
2.5 SimpleNonparametricRegression .................................. 44
2.6 AdditiveModels...................................................... 49
2.7 GeneralizedAdditiveModels........................................ 52
2.8 GeoadditiveRegression.............................................. 55
2.9 BeyondMeanRegression............................................ 61
2.9.1 RegressionModelsforLocation,Scale,andShape....... 62
2.9.2 QuantileRegression........................................ 66
2.10 ModelsinaNutshell................................................. 68
2.10.1 LinearModels(LMs,Chaps.3and4)..................... 68
2.10.2 LogitModel(Chap.5)...................................... 68
2.10.3 PoissonRegression(Chap.5).............................. 68
2.10.4 GeneralizedLinearModels(GLMs,Chaps.5and6)..... 69
2.10.5 LinearMixedModels(LMMs,Chap.7)................... 69
2.10.6 AdditiveModelsand Extensions(AMs,
Chaps.8and9).............................................. 70
2.10.7 Generalized Additive (Mixed) Models
(GA(M)Ms,Chap.9) ....................................... 70
2.10.8 StructuredAdditiveRegression(STAR,Chap.9)......... 71
2.10.9 QuantileRegression(Chap.10)............................ 71
ix