Table Of Content

Statistics for Biology and Health Series Editors: M. Gail K. Krickeberg J. Samet A. Tsiatis W. Wong For other titles published in this series, go to http://www.springer.com/series/2848 Per Kragh Andersen • Lene Theil Skovgaard Regression with Linear Predictors With 171 illustrations by Therese Graversen 1 C Per Kragh Andersen Lene Theil Skovgaard University of Copenhagen University of Copenhagen Dept. Biostatistics Dept. Biostatistics Øster Farimagsgade 5 Øster Farimagsgade 5 DK-1014 Copenhagen K DK-1014 Copenhagen K Denmark Denmark [email protected] [email protected] Series Editors M. Gail A. Tsiatis National Cancer Institute Department of Statistics Bethesda, MD 20892, USA North Carolina State University Raleigh, NC 27695, USA K. Krickeberg Le Chatelet W. Wong F-63270 Manglieu, France Department of Statistics Stanford University J. Samet Stanford, CA 94305-4065, USA Department of Preventive Medicine Keck School of Medicine University of Southern California 1441 Eastlake Ave. Room 4436, MC 9175 Los Angeles, CA 90089 ISSN 1431-8776 ISBN 978-1-4419-7169-2 e-ISBN 978-1-4419-7170-8 DOI 10.1007/978-1-4419-7170-8 Springer New York Dordrecht Heidelberg London Library of Congress Control Number: 2010931468 © Springer Science+Business Media, LLC 2010 All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com) V Preface This is a book about regression analysis, that is, the situation in statistics where the distribution of a response (or outcome) variable is related to explanatory variables (or covariates). This is an extremely common situation in theapplicationofstatisticalmethodsinmanyfields,andlinearregression,logistic regression, and Cox proportional hazards regression are frequently used for quantitative, binary, and survival time outcome variables, respectively. Several books on these topics have appeared and for that reason one may well ask why we embark on writing still another book on regression. We have two main reasons for doing this: 1. First,wewanttohighlightsimilaritiesamonglinear,logistic,proportional hazards,andotherregressionmodelsthatincludealinearpredictor.These modelsareoftentreatedentirelyseparatelyintextsinspiteofthefactthat alloperationsonthemodelsdealingwiththelinearpredictorareprecisely the same, including handling of categorical and quantitative covariates, testing for linearity and studying interactions. 2. Second, we want to emphasize that, for any type of outcome variable, multiple regression models are composed of simple building blocks that areaddedtogetherinthelinearpredictor:thatis,t-tests,one-wayanalyses of variance and simple linear regressions for quantitative outcomes, 2×2, 2×(k+1)tables andsimplelogistic regressionsfor binaryoutcomes, and 2-and(k+1)-samplelogranktestsandsimpleCoxregressionsforsurvival data.Thishastwoconsequences.Allthesesimpleandwellknownmethods can be considered as special cases of the regression models. On the other hand, the effect of a single explanatory variable in a multiple regression model can be interpreted in a way similar to that obtained in the simple analysis, however, now valid only for the other explanatory variables in the model “held fixed”. Note the important point that addition of simple terms in the linear predictor will imply an assumption of no interaction; that is, the effect of an explanatory variable is the same for all values of otherexplanatoryvariablesinthemodel.Thisisanassumptionthatoften needs careful consideration as part of the analysis. In Chapter 1 the basic ideas are set up and the examples to be used throughout the book are introduced. Chapter 2 presents a review of background material on probability distributions and the principles of statistical inference. In Chapter 3 the simple building blocks for categorical explanatory variables are introduced for the three main types of outcome variables. Chapter 4 deals with one quantitative explanatory variable, first when a linear effect can be assumed and, next, certain models with a nonlinear effect of the covariate (still described by a linear predictor, however) are introduced. A very common example of such a nonlinear effect is a polynomial. VI Havingpresentedthebuildingblocks,Chapter5discussesmultipleregres- sion models (i.e., models with several explanatory variables). We focus on models with two covariates and introduce the concepts of confounding and interaction (“effect modification”). In Chapter 6 we discuss model building strategies,inparticularselectionofexplanatoryvariablesforansweringaspe- cific research question and illustrate the strategies by thorough analyses of three specific examples. WhereasChapters3through6primarilydealwithexamplesoflinearmod- elsforquantitativeoutcomes,logisticmodelsforbinarydata,andCoxmodels for lifetimes, Chapter 7 presents a number of other regression models with a linear predictor. These include the logistic models for ordinal and multinomial data, Poisson-type models for counts as well as alternative models for quantitative, binary, and lifetime data. Chapter 8 briefly mentions a number of extended models all involving a linear predictor. These include multivariate models with more than one response variable per individual, for example repeated measurements and other types of correlated outcomes, and models with covariate measurement errors. The multivariate models include random effect models and marginal models. The treatment of these topics is by no means meant to be exhaustive but to serve mainly as a warning that models more complicated than the ones treated earlier in this book occur quite frequently and will produce erroneous conclusions if not analyzed properly. Thebookisconcludedwithfourappendicessummarizingnotation,theuseof logarithms, some recommendations, and simple programming, respectively. It is important to notice that some sections of the book are more difficult to read than others, simply because of the varying level of complexity for the different methods we wish to cover. Thebookisbasedonourpersonalexperienceasteachers,consultants,and researchersinbiostatistics formorethanthreedecadesandalldataexamples are based on this. Our intended readers are primarily researchers from scientific areas where statistics is being applied to analyze numerical data, for example, fields such as medicine, public health, dentistry, agriculture, and so on. For that reason we do not expect readers to have a strong background in mathematics. We limit the amount of mathematical formulas used, and we avoid the use of Greek letters in formulas altogether. We do, however, expect readers to have some familiarity with basic statistical terms but, to set a common level, Chapter 2 gives an overview of the necessary concepts. Although we have mainlywrittenthebookforappliedscientistsitisourhopethatreaderswith a more mathematical background who wish to enter the field of biostatistics willalsobenefitfromstudyingthebook.Weintendtoprovideabookthatcan both serve as a reference source and a course textbook. For the last purpose, chapters conclude with a series of exercises all dealing with data analysis. The mainstream of our text presents aspects of the methods that are important for building regression models, yet we have in some places found it natural to add brief discussions of related concepts that are less important VII for regression analysis. Such digressions are marked in the text as Digression and also include some historical and more technical remarks. An applied statistical text such as the present book, obviously includes a large number of practical examples, and computational aspects are crucial. We have chosen not to base the presentation of examples in the book on a single piece of software. Instead, the book is accompanied by Web pages documenting examples by including computer code for the computations in R and SAS. It is our intention to supplement the Web pages with code in STATA as well but we have chosen not to include SPSS code because it is our impressionthatmostusersofthisprogramusethemenuinterfaceratherthan writing program code. In addition, a brief appendix (Appendix D) includes very simple and “raw” commands for fitting the basic types of regression models in R, SAS, and STATA. It has been our ambition that the Web pages should be user-friendly with facilitiesformakinganentrybasedonbothbookchaptersandonthedifferent examples. The pages can be found at www.biostat.ku.dk/~linearpredictors We would like to thank our medical colleagues who have granted us permission to use their data as illustrations. We also wish to thank colleagues at theDepartmentofBiostatistics,UniversityofCopenhagen,Denmark,forcre- atingafriendlyandproductiveworkingenvironment. Inparticular,wethank ØrnulfBorgan,BendixCarstensen,SaskialeCessie,ThomasGerds,NielsKei- ding, Kajsa Kvist, Maja Olsbjerg Larsen, Henrik Ravn and Willi Sauerbrei for comments, advice, assistance or encouragement. Part of the manuscript was written during a week-long workshop in Oberwolfach, Germany, and we wish to thank Mathematisches Forschungsinstitut, Oberwolfach, for hospital- ity during that week. Last, but definitely not least, our most sincere thanks go to Therese Gra- versen. Not only did she design the accompanying Web pages and produce all the graphics in the manuscript but she also gave valuable and thoughtful comments to both contents and lay-out. Without her skillful input this book would never have become what it is now. Copenhagen,June2010 PerKraghAndersen,LeneTheilSkovgaard Contents 1 Introduction............................................... 1 1.1 Introductory examples and types of outcome................ 3 1.1.1 Introductory examples ............................. 3 1.1.2 Types of outcome ................................. 7 1.2 Covariates.............................................. 9 1.2.1 Categorical covariates.............................. 9 1.2.2 Quantitative covariates............................. 11 1.3 Link functions .......................................... 13 1.4 Building a regression model............................... 22 1.4.1 The linear predictor and the link function ............ 24 1.4.2 Regression models and their interpretation ........... 28 1.5 Further examples........................................ 30 1.6 The scope of this book and how to read it .................. 37 2 Statistical models.......................................... 43 2.1 Random variables and probability ......................... 44 2.1.1 The Bernoulli distribution .......................... 47 2.1.2 The Binomial distribution .......................... 48 2.1.3 The Poisson distribution ........................... 50 2.1.4 The Normal distribution ........................... 51 2.1.5 Other common distributions ........................ 55 2.1.6 Conditional probability ............................ 56 2.2 Descriptive statistics..................................... 57 2.2.1 Binary outcome ................................... 57 2.2.2 Quantitative outcome.............................. 59 2.2.3 Survival time outcome ............................. 63 2.3 Statistical inference...................................... 64 2.3.1 Estimation ....................................... 65 2.3.2 Model checking ................................... 73 2.3.3 Hypothesis testing................................. 79 2.3.4 The likelihood function............................. 84 X Contents 2.4 Exercises ............................................... 92 3 One categorical covariate .................................. 95 3.1 Binary covariate......................................... 96 3.1.1 Quantitative outcome: t-tests ....................... 97 3.1.2 Binary outcome: (2×2)-tables and the chi-square test ..110 3.1.3 Survival time outcome: the 2-sample logrank test......123 3.2 Categorical covariate with more than two levels .............137 3.2.1 Quantitative outcome: One-way analysis of variance ...142 3.2.2 Binary outcome: The 2×(k+1)-table ...............157 3.2.3 Survival time outcome: The (k+1)-sample logrank test 161 3.3 Exercises ...............................................166 4 One quantitative covariate.................................173 4.1 Linear effect ............................................175 4.1.1 Quantitative outcome: Simple linear regression ........176 4.1.2 Binary outcome: Simple logistic regression............194 4.1.3 Survival time outcome: Simple Cox regression.........201 4.2 Nonlinear effect .........................................210 4.2.1 Dividing the covariate range into intervals ............214 4.2.2 Polynomials ......................................221 4.2.3 Other nonlinear models with a linear predictor ........223 4.3 Exercises ...............................................228 5 Multiple regression, the linear predictor ...................231 5.1 Two covariates: Models without interaction .................234 5.1.1 Two categorical covariates..........................234 5.1.2 One categorical and one quantitative covariate ........245 5.1.3 Two quantitative covariates.........................254 5.2 Two covariates: Models with interaction....................263 5.2.1 Two categorical covariates..........................264 5.2.2 One categorical and one quantitative covariate ........273 5.2.3 Two quantitative covariates.........................278 5.2.4 Saving degrees-of-freedom ..........................285 5.3 Several covariates........................................287 5.3.1 Models without higher-order interactions .............287 5.3.2 Models with higher-order interactions ................289 5.4 Matched studies.........................................294 5.5 Exercises ...............................................300 6 Model building: From purpose to conclusion...............303 6.1 General principles for model selection ......................304 6.1.1 Identification of covariates..........................305 6.1.2 Model diagrams ...................................308 6.1.3 Initial model building ..............................311 Contents XI 6.1.4 Strategy of analysis................................314 6.1.5 Model checks and diagnostics .......................317 6.1.6 Collinearity.......................................318 6.1.7 Interactions.......................................319 6.2 Examples...............................................320 6.2.1 The vitamin D example ............................321 6.2.2 The surgery example...............................331 6.2.3 The PBC-3 trial...................................342 6.3 Sample size determination ................................354 6.4 Exercises ...............................................365 7 Alternative outcome types and link functions..............367 7.1 Multinomial outcome ....................................367 7.1.1 Ordinal outcome ..................................368 7.1.2 Nominal outcome..................................383 7.2 Count outcome..........................................387 7.3 Quantitative outcome ....................................394 7.4 Binary outcome .........................................403 7.4.1 Alternatives to the logit link ........................404 7.4.2 Case-control studies ...............................409 7.5 Survival time outcome ...................................416 7.5.1 Multiplicative hazard models .......................416 7.5.2 Additive hazard models ............................422 7.5.3 Accelerated failure time models .....................427 7.6 Exercises ...............................................429 8 Further topics .............................................431 8.1 Multivariate outcome ....................................431 8.1.1 Random effects models.............................433 8.1.2 Marginal models ..................................438 8.1.3 Longitudinal and life history data ...................440 8.2 Errors in covariates ......................................447 8.2.1 Regression dilution ................................448 8.2.2 Correction for measurement error in covariates ........452 A Appendix: Notation .......................................457 B Appendix: Use of logarithms...............................463 C Appendix: Some recommendations.........................473 D Programming in R, SAS and STATA...........................477 References.....................................................483 Index..........................................................487 1 Introduction Supposewearestudyingbloodpressureinhumansbasedonarandomsample from a specific population, say, inhabitants of some larger city. The very first step in such a study may be to get a summary of the level and variation of blood pressure, subject to criteria such as ethnicity, gender or age. The purpose of studying blood pressure may be to establish normal references to serve as future guidelines for when to start treatment for either too high or too low a blood pressure. In order to illustrate the distribution of the blood pressure measurements from this sample, we usually calculate average and standard deviation and possiblyproduceahistogramorsomeothergraphicalillustration.Thesequan- tities are examples of descriptive statistics. Asmallpartofthevariationinthebloodpressuremeasurementscanprob- ablybeascribedtomeasurementerror,however,alargerpartofthevariation is more likely due to individual fluctuations over time and topopulation variation, the true differences between subjects. Part of this population variation may be due to characteristics of the subjects that are easily recognized. For instance,menfrequentlyhaveasomewhathigherbloodpressurethanwomen, and older people tend to have a higher blood pressure than younger people. The example illustrates that, in many fields of research, information may be collected on various features in a number of experimental units. Other examples may be the serum bilirubin level in male and female patients with liver cirrhosis, whether persons in different job categories working in a given company have suffered from severe headaches in some specified period, the survival time from diagnosis of cancer patients in different stages of disease, the rise in blood glucose in experimental animals after feeding with different diets, the number of claims in a year for insurance policies of different types, the yield of some crop in differently treated field plots in an agricultural experiment, or the annual cancer rates in successive years in some country. Inthelastthreeexamplestheexperimentalunitsaretheinsurancepolicy, the field plot, and the country. However, in this book we are mainly using examples from medical and public health research and we denote the experi-