Multiple Regression and Beyond: An Introduction to Multiple Regression and Structural Equation Modeling

605 Pages·2015·4.06 MB·English

by  Timothy Z. Keith

Multiple Regression
and Beyond
Multiple Regression and Beyond offers a conceptually oriented introduction to multiple
regression (MR) analysis and structural equation modeling (SEM), along with analyses that
fl ow naturally from those methods. By focusing on the concepts and purposes of MR and
related methods, rather than the derivation and calculation of formulae, this book introduces
material to students more clearly, and in a less threatening way. In addition to illuminating
content necessary for coursework, the accessibility of this approach means students are more
likely to be able to conduct research using MR or SEM—and more likely to use the methods
wisely.
• Covers both MR and SEM, while explaining their relevance to one another
• Also includes path analysis, confirmatory factor analysis, and latent growth modeling
• Figures and tables throughout provide examples and illustrate key concepts and
techniques
Timothy Z. Keith is Professor and Program Director of School Psychology at University of
Texas, Austin.
This page intentionally left blank
Multiple Regression
and Beyond
An Introduction to
Multiple Regression
and Structural
Equation Modeling
2nd Edition
Timothy Z. Keith
Second edition published 2015
by Routledge
711 Third Avenue, New York, NY 10017
and by Routledge
2 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN
Routledge is an imprint of the Taylor & Francis Group, an informa business
© 2015 Taylor & Francis
The right of Timothy Z. Keith to be identified as author of this work has been asserted by
him in accordance with sections 77 and 78 of the Copyright, Designs and Patents Act 1988.
All rights reserved. No part of this book may be reprinted or reproduced or utilized in any
form or by any electronic, mechanical, or other means, now known or hereafter invented,
including photocopying and recording, or in any information storage or retrieval system,
without permission in writing from the publishers.
Trademark Notice: Product or corporate names may be trademarks or registered
trademarks, and are used only for identification and explanation without intent to infringe.
First edition published by Pearson Education, Inc. 2006
Library of Congress Cataloging-in-Publication Data
Library of Congress Control Number: 2014956124
ISBN: 978-1-138-81194-2 (hbk)
ISBN: 978-1-138-81195-9 (pbk)
ISBN: 978-1-315-74909-9 (ebk)
Typeset in Minion
by Apex CoVantage, LLC
Contents
Preface vii
Acknowledgments xi
Part I Multiple Regression 1
1 Introduction: Simple (Bivariate) Regression 3
2 Multiple Regression: Introduction 26
3 Multiple Regression: More Detail 44
4 Three and More Independent Variables and Related Issues 57
5 Three Types of Multiple Regression 77
6 Analysis of Categorical Variables 108
7 Categorical and Continuous Variables 129
8 Continuous Variables: Interactions and Curves 161
9 Multiple Regression: Summary, Assumptions, Diagnostics,
Power, and Problems 182
10 Related Methods: Logistic Regression and Multilevel Modeling 213
Part II Beyond Multiple Regression: Structural Equation Modeling 241
11 Path Modeling: Structural Equation Modeling
With Measured Variables 243
v
vi (cid:129) CONTENTS
12 Path Analysis: Dangers and Assumptions 267
13 Analyzing Path Models Using SEM Programs 282
14 Error: The Scourge of Research 318
15 Confirmatory Factor Analysis I 332
16 Putting It All Together: Introduction to Latent Variable SEM 371
17 Latent Variable Models: More Advanced Topics 391
18 Latent Means in SEM 424
19 Confirmatory Factor Analysis II: Invariance and Latent Means 455
20 Latent Growth Models 493
21 Summary: Path Analysis, CFA, SEM, and Latent Growth Models 514
Appendices
Appendix A: Data Files 537
Appendix B: Review of Basic Statistics Concepts 539
Appendix C: Partial and Semipartial Correlation 557
Appendix D: Symbols Used in This Book 565
Appendix E: Useful Formulae 567
References 569
Author Index 579
Subject Index 583
Preface
Multiple Regression and Beyond is designed to provide a conceptually oriented introduction to
multiple regression along with more complex methods that fl ow naturally from multiple regres-
sion: path analysis, confi rmatory factor analysis, and structural equation modeling. Multiple
regression (MR) and related methods have become indispensable tools for modern social science
researchers. MR closely implements the general linear model and thus subsumes methods, such
as analysis of variance (ANOVA), that have traditionally been more commonplace in psycho-
logical and educational research. Regression is especially appropriate for the analysis of nonex-
perimental research, and with the use of dummy variables and modern computer packages, it is
often more appropriate or easier to use MR to analyze the results of complex quasi-experimental
or even experimental research. Extensions of multiple regression—particularly structural equa-
tion modeling (SEM)—partially obviate threats due to the unreliability of the variables used in
research and allow the modeling of complex relations among variables. A quick perusal of the full
range of social science journals demonstrates the wide applicability of the methods.
Despite its importance, MR-based analyses are too often poorly conducted and poorly
reported. I believe one reason for this incongruity is inconsistency between how material is
presented and how most students best learn.
Anyone who teaches (or has ever taken) courses in statistics and research methodology
knows that many students, even those who may become gifted researchers, do not always
gain conceptual understanding through numerical presentation. Although many who teach
statistics understand the processes underlying a sequence of formulas and gain conceptual
understanding through these formulas, many students do not. Instead, such students often
need a thorough conceptual explanation to gain such understanding, after which a numeri-
cal presentation may make more sense. Unfortunately, many multiple regression textbooks
assume that students will understand multiple regression best by learning matrix algebra,
wading through formulas, and focusing on details.
At the same time, methods such as structural equation modeling (SEM) and confirma-
tory factor analysis (CFA) are easily taught as extensions of multiple regression. If structured
properly, multiple regression flows naturally into these more complex topics, with nearly
complete carry-over of concepts. Path models (simple SEMs) illustrate and help deal with
some of the problems of MR, CFA does the same for path analysis, and latent variable SEM
combines all the previous topics into a powerful, flexible methodology.
I have taught courses including these topics at four universities (the University of Iowa,
Virginia Polytechnic Institute & State University, Alfred University, and the University of
vii
viii (cid:129) PREFACE
Texas). These courses included faculty and students in architecture, engineering, educational
psychology, educational research and statistics, kinesiology, management, political science,
psychology, social work, and sociology, among others. This experience leads me to believe
that it is possible to teach these methods by focusing on the concepts and purposes of MR
and related methods, rather than the derivation and calculation of formulas (what my wife
calls the “plug and chug” method of learning statistics). Students generally find such an
approach clearer, more conceptual, and less threatening than other approaches. As a result
of this conceptual approach, students become interested in conducting research using MR,
CFA, or SEM and are more likely to use the methods wisely.
THE ORIENTATION OF THIS BOOK
My overriding bias in this book is that these complex methods can be presented and learned
in a conceptual, yet rigorous, manner. I recognize that not all topics are covered in the depth
or detail presented in other texts, but I will direct you to other sources for topics for which
you may want additional detail. My style is also fairly informal; I’ve written this book as if I
were teaching a class.
Data
I also believe that one learns these methods best by doing, and the more interesting and rel-
evant that “doing,” the better. For this reason, there are numerous example analyses through-
out this book that I encourage you to reproduce as you read. To make this task easier, the Web
site that accompanies the book (www.tzkeith.com) includes the data in a form that can be
used in common statistical analysis programs. Many of the examples are taken from actual
research in the social sciences, and I’ve tried to sample from research from a variety of areas.
In most cases simulated data are provided that mimic the actual data used in the research.
You can reproduce the analyses of the original researchers and, perhaps, improve on them.
And the data feast doesn’t end there! The Web site also includes data from a major federal
data set: 1000 cases from the National Education Longitudinal Study (NELS) from the National
Center for Education Statistics. NELS was a nationally representative sample of 8th-grade stu-
dents first surveyed in 1988 and resurveyed in 10th and 12th grades and then twice after leav-
ing high school. The students’ parents, teachers, and school administrators were also surveyed.
The Web site includes student and parent data from the base year (8th grade) and student data
from the first follow-up (10th grade). Don’t be led astray by the word Education in NELS; the
students were asked an incredible variety of questions, from drug use to psychological well-
being to plans for the future. Anyone with an interest in youth will find something interesting
in these data. Appendix A includes more information about the data at www.tzkeith.com.
Computer Analysis
Finally, I fi rmly believe that any book on statistics or research methods should be closely related
to statistical analysis software. Why plug and chug—plug numbers into formulas and chug out
the answers on a calculator—when a statistical program can do the calculations more quickly
and accurately with, for most people, no loss of understanding? Freed from the drudgery of
hand calculations, you can then concentrate on asking and answering important research ques-
tions, rather than on the intricacies of calculating statistics. This bias toward computer calcu-
lations is especially important for the methods covered in this book, which quickly become
unmanageable by hand. Use a statistical analysis program as you read this book; do the exam-
ples with me and the problems at the end of the chapters, using that program.
Which program? I use SPSS as my general statistical analysis program, and you can get
the program for a reasonable price as a student in a university (approximately $100–$125
PREFACE (cid:129) ix
per year for the “Grad Pack” as this is written). But you need not use SPSS; any of the com-
mon packages will do (e.g., SAS or SYSTAT). The output in the text has a generic look to it,
which should be easily translatable to any major statistical package output. In addition, the
website (www.tzkeith.com) includes sample multiple regression and SEM output from vari-
ous statistical packages.
For the second half of the book, you will need access to a structural equation modeling
program. Fortunately, student or tryout versions of many such programs are available online.
Student pricing for the program used extensively in this book, Amos, is available, at this writ-
ing, for approximately $50 per year as an SPSS add-on. Although programs (and pricing)
change, one current limitation of Amos is that there is no Mac OS version of Amos. If you
want to use Amos, you need to be able to run Windows. Amos is, in my opinion, the easiest
SEM program to use (and it produces really nifty pictures). The other SEM program that
I will frequently reference is Mplus. We’ll talk more about SEM in Part 2 of this book. The
website for this text has many examples of SEM input and output using Amos and Mplus.
Overview of the Book
This book is divided into two parts. Part 1 focuses on multiple regression analysis. We begin
by focusing on simple, bivariate regression and then expand that focus into multiple regres-
sion with two, three, and four independent variables. We will concentrate on the analysis
and interpretation of multiple regression as a way of answering interesting and important
research questions. Along the way, we will also deal with the analytic details of multiple
regression so that you understand what is going on when we do a multiple regression analysis.
We will focus on three different types, or fl avors, of multiple regression that you will encoun-
ter in the research literature, their strengths and weaknesses, and their proper interpretation.
Our next step will be to add categorical independent variables to our multiple regression
analyses, at which point the relation of multiple regression and ANOVA will become clearer.
We will learn how to test for interactions and curves in the regression line and to apply these
methods to interesting research questions.
The penultimate chapter for Part 1 is a review chapter that summarizes and integrates
what we have learned about multiple regression. Besides serving as a review for those who
have gone through Part 1, it also serves as a useful introduction for those who are interested
primarily in the material in Part 2. In addition, this chapter introduces several important
topics not covered completely in previous chapters. The final chapter in Part 1 presents two
related methods, logistic regression and multilevel modeling, in a conceptual fashion using
what we have learned about multiple regression.
Part 2 focuses on structural equation modeling—the “Beyond” portion of the book’s
title. We begin by discussing path analysis, or structural equation modeling with measured
variables. Simple path analyses are easily estimated via multiple regression analysis, and
many of our questions about the proper use and interpretation of multiple regression will
be answered with this heuristic aid. We will deal in some depth with the problem of valid
versus invalid inferences of causality in these chapters. The problem of error (“the scourge of
research”) serves as our jumping off place for the transition from path analysis to methods
that incorporate latent variables (confirmatory factor analysis and latent variable structural
equation modeling). Confirmatory factor analysis (CFA) approaches more closely the con-
structs of primary interest in our research by separating measurement error from variation
due to these constructs. Latent variable structural equation modeling (SEM) incorporates
the advantages of path analysis with those of confirmatory factor analysis into a powerful
and flexible analytic system that partially obviates many of the problems we discuss as the
book progresses. As we progress to more advanced SEM topics we will learn how to test for