Regression Analysis by Example

421 Pages·2012·29.56 MB·English

by  Samprit Chatterjee

Regession Analysis
By Example
WILEY SERIES IN PROBABILITY AND STATISTICS
Established by WALTER A. SHEWHART and SAMUEL S. WILKS
Editors: David J. Balding, Noel A. C. Cressie, Garrett M. Fitzmaurice,
Harvey Goldstein, lain M. Johnstone, Geert Molenberghs, David W. Scott,
Adrian F M. Smith, Ruey S. Tsay, Sanford Weisberg
Editors Emeriti: Vic Barnett, J. Stuart Hunter, Joseph B. Kadane, Jozef L. Teugels
A complete list of the titles in this series appears at the end of this volume.
Regression Analysis
By Example
Fifth Edition
Samprit Chatterjee
Ali S. Hadi
(f)WILEY
A JOHN WILEY & SONS, INC., PUBLICATION
Copyright © 2012 by John Wiley & Sons, Inc. All rights reserved
Published by John Wiley & Sons, Inc., Hoboken, New Jersey
Published simultaneously in Canada
No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form
or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as
permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior
written permission of the Publisher, or authorization through payment ofthe appropriate per-copy fee to
the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, [phone], fax
[phone], or on the web at www.copyright.com. Requests to the Publisher for permission should
be addressed to the Permissions Department, John Wiley & Sons, Inc., III River Street, Hoboken, NJ
07030, [phone], fax [phone], or online at http://www.wiley.comlgo/permission.
Limit of LiabilitylDisclaimer of Warranty: While the publisher and author have used their best efforts in
preparing this book, they make no representations or warranties with respect to the accuracy or
completeness of the contents of this book and specifically disclaim any implied warranties of
merchantability or fitness for a particular purpose. No warranty may be created or extended by sales
representatives or written sales materials. The advice and strategies contained herein may not be
suitable for your situation. You should consult with a professional where appropriate. Neither the
publisher nor author shall be liable for any loss of profit or any other commercial damages, including
but not limited to special, incidental, consequential, or other damages.
For general information on our other products and services or for technical support, please contact our
Customer Care Department within the United States at [phone], outside the United States at
[phone] or fax [phone].
Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may
not be available in electronic formats. For more information about Wiley products, visit our web site at
www.wiley.com.
Library of Congress Cataloging-in-Publication Data:
Chatterjee, Samprit, 1938-
Regression analysis by example I Samprit Chatterjee, New York University, Leonard N. Stern School
of Business, New York, NY, Ali S. Hadi, American University in Cairo, Actuarial Science Program,
Cairo, Egypt. - Fifth edition.
pages cm. - (Wiley series in probability and statistics)
Includes index.
ISBN 978-0-470-90584-5 (hardback)
I. Regression analysis. I. Hadi, Ali S. II. Title.
QA278.2.C5 2006
519.5'36-dc23 2012009795
Printed in the United States of America.
10 9 8 7 6 5 4 3 2 I
Dedicated to:
c.
Allegra, Martha, and Rima - S.
The memory of my parents - A. S. H.
It's a gift to be simple ...
Old Shaker hymn
True knowledge is knowledge of why things are
as they are, and not merely what they are.
Isaiah Berlin

CONTENTS
Preface xiii
1 Introduction 1
1.1 What Is Regression Analysis? 1
1.2 Publicly Available Data Sets 2
1.3 Selected Applications of Regression Analysis 3
1.3.1 Agricultural Sciences 3
1.3.2 Industrial and Labor Relations 4
1.3.3 Government 6
1.3.4 History 6
1.3.5 Environmental Sciences 9
1.3.6 Industrial Production 9
1.3.7 The Space Shuttle Challenger 12
1.3.8 Cost of Health Care 12
1.4 Steps in Regression Analysis 13
1.4.1 Statement of the Problem 13
1.4.2 Selection of Potentially Relevant Variables 15
1.4.3 Data Collection 15
1.4.4 Model Specification 16
1.4.5 Method of Fitting 19
1.4.6 Model Fitting 19
1.4.7 Model Criticism and Selection 19
1.4.8 Objectives of Regression Analysis 20
vii
viii CONTENTS
1.5 Scope and Organization of the Book 21
Exercises 23
2 Simple Linear Regression 25
2.1 Introduction 25
2.2 Covariance and Correlation Coefficient 25
2.3 Example: Computer Repair Data 30
2.4 The Simple Linear Regression Model 32
2.5 Parameter Estimation 33
2.6 Tests of Hypotheses 36
2.7 Confidence Intervals 41
2.8 Predictions 41
2.9 Measuring the Quality of Fit 43
2.10 Regression Line Through the Origin 46
2.11 Trivial Regression Models 48
2.12 Bibliographic Notes 49
Exercises 49
3 Multiple Linear Regression 57
3.1 Introduction 57
3.2 Description of the Data and Model 57
3.3 Example: Supervisor Performance Data 58
3.4 Parameter Estimation 59
3.5 Interpretations of Regression Coefficients 62
3.6 Centering and Scaling 64
3.6.1 Centering and Scaling in Intercept Models 65
3.6.2 Scaling in No-Intercept Models 66
3.7 Properties of the Least Squares Estimators 67
3.8 Multiple Correlation Coefficient 68
3.9 Inference for Individual Regression Coefficients 69
3.10 Tests of Hypotheses in a Linear Model 71
3.10.1 Testing All Regression Coefficients Equal to Zero 73
3.10.2 Testing a Subset of Regression Coefficients Equal to
Zero 75
3.10.3 Testing the Equality of Regression Coefficients 78
3.10.4 Estimating and Testing of Regression Parameters
Under Constraints 79
3.11 Predictions 81
3.12 Summary 82
Exercises 82
Appendix: Multiple Regression in Matrix Notation 89