Multiple Regression and Beyond: An Introduction to Multiple Regression and Structural Equation Modeling

654 Pages·2019·10.82 MB·English

by  Timothy Z. Keith

Multiple Regression
and Beyond
Multiple Regression and Beyond offers a conceptually-oriented introduction to multiple
regression (MR) analysis and structural equation modeling (SEM), along with analyses that
flow naturally from those methods. By focusing on the concepts and purposes of MR and
related methods, rather than the derivation and calculation of formulae, this book introduces
material to students more clearly, and in a less threatening way. In addition to illuminating
content necessary for coursework, the accessibility of this approach means students are more
likely to be able to conduct research using MR or SEM—and more likely to use the methods
wisely.
This book:
• Covers both MR and SEM, while explaining their relevance to one another
• Includes path analysis, confirmatory factor analysis, and latent growth modeling
• Makes extensive use of real-world research examples in the chapters and in the
end-of-chapter exercises
• Extensive use of figures and tables providing examples and illustrating key concepts
and techniques
New to this edition:
• New chapter on mediation, moderation, and common cause
• New chapter on the analysis of interactions with latent variables and multilevel SEM
• Expanded coverage of advanced SEM techniques in chapters 18 through 22
• International case studies and examples
• Updated instructor and student online resources
Timothy Z. Keith is Professor of Educational Psychology at the University of Texas, Austin.
His research is focused on the nature and measurement of intelligence, including the validity
of tests of intelligence and the theories from which they are drawn. His research has been
recognized with awards from the three major journals in school psychology, and he was
awarded the senior scientist distinction by the School Psychology division of APA.

Multiple Regression
and Beyond
An Introduction to
Multiple Regression
and Structural
Equation Modeling
Third Edition
Timothy Z. Keith
Third edition published 2019
by Routledge
52 Vanderbilt Avenue, New York, NY 10017
and by Routledge
2 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN
Routledge is an imprint of the Taylor & Francis Group, an informa business
© 2019 Taylor & Francis
The right of Timothy Z. Keith to be identified as author of this work has been asserted by
him in accordance with sections 77 and 78 of the Copyright, Designs and Patents Act 1988.
All rights reserved. No part of this book may be reprinted or reproduced or utilised in any
form or by any electronic, mechanical, or other means, now known or hereafter invented,
including photocopying and recording, or in any information storage or retrieval system,
without permission in writing from the publishers.
Trademark notice: Product or corporate names may be trademarks or registered
trademarks, and are used only for identification and explanation without intent to infringe.
First edition published by Pearson Education 2006
Second edition published by Routledge 2014
Library of Congress Cataloging-in-Publication Data
Names: Keith, Timothy Z., author.
Title: Multiple regression and beyond : an introduction to multiple regression
and structural equation modeling / Timothy Z. Keith.
Description: Third Edition. | New York : Routledge, 2019. | Revised edition
of the author’s Multiple regression and beyond, 2015.
Identifiers: LCCN 2018041116 | ISBN 9781138061422 (hardback) |
ISBN 9781138061446 (pbk.) | ISBN 9781315162348 (ebook)
Subjects: LCSH: Regression analysis.
Classification: LCC HA31.3 .K45 2019 | DDC 519.5/36—dc23
LC record available at https://lccn.loc.gov/2018041116
ISBN: 978-1-138-06142-2 (hbk)
ISBN: 978-1-138-06144-6 (pbk)
ISBN: 978-1-315-16234-8 (ebk)
Typeset in Minion
by Apex CoVantage, LLC
Visit the companion website: www.tzkeith.com
Contents
Preface vii
Acknowledgments xiii
Part I Multiple Regression 1
1 Simple Bivariate Regression 3
2 Multiple Regression: Introduction 26
3 Multiple Regression: More Depth 44
4 Three and More Independent Variables and Related Issues 57
5 Three Types of Multiple Regression 77
6 Analysis of Categorical Variables 108
7 Regression With Categorical and Continuous Variables 129
8 Testing for Interactions and Curves With Continuous Variables 161
9 Mediation, Moderation, and Common Cause 177
10 Multiple Regression: Summary, Assumptions, Diagnostics,
Power, and Problems 195
11 Related Methods: Logistic Regression and Multilevel Modeling 226
v
vi • Contents
Part II Beyond Multiple Regression: Structural Equation Modeling 255
12 Path Modeling: Structural Equation Modeling With Measured
Variables 257
13 Path Analysis: Assumption and Dangers 281
14 Analyzing Path Models Using SEM Programs 296
15 Error: The Scourge of Research 334
16 Confirmatory Factor Analysis I 348
17 Putting It All Together: Introduction to Latent Variable SEM 389
18 Latent Variable Models II: Multigroup Models, Panel Models,
Dangers and Assumptions 409
19 Latent Means in SEM 444
20 Confirmatory Factor Analysis II: Invariance and Latent Means 475
21 Latent Growth Models 513
22 Latent Variable Interactions and Multilevel Modelling in SEM 534
23 Summary: Path Analysis, CFA, SEM, Mean Structures, and Latent
Growth Models 561
Appendices
Appendix A: Data Files 585
Appendix B: Review of Basic Statistics Concepts 587
Appendix C: Partial and Semipartial Correlation 605
Appendix D: Symbols Used in This Book 613
Appendix E: Useful Formulae 615
References 617
Author Index 629
Subject Index 633
Preface
Multiple Regression and Beyond is designed to provide a conceptually oriented introduction
to multiple regression along with more complex methods that flow naturally from multiple
regression: path analysis, confirmatory factor analysis, and structural equation modeling.
Multiple regression (MR) and related methods have become indispensable tools for modern
social science researchers. MR closely implements the general linear model and thus sub-
sumes methods, such as analysis of variance (ANOVA), that have traditionally been more
commonplace in psychological and educational research. Regression is especially appropri-
ate for the analysis of nonexperimental research, and with the use of dummy variables and
modern computer packages, it is often more appropriate or easier to use MR to analyze the
results of complex quasi-experimental or even experimental research. Extensions of multiple
regression—particularly structural equation modeling (SEM)—partially obviate threats due
to the unreliability of the variables used in research and allow the modeling of complex rela-
tions among variables. A quick perusal of the full range of social science journals demon-
strates the wide applicability of the methods.
Despite its importance, MR-based analyses are too often poorly conducted and poorly
reported. I believe one reason for this incongruity is inconsistency between how material is
presented and how most students best learn.
Anyone who teaches (or has ever taken) courses in statistics and research methodology
knows that many students, even those who may become gifted researchers, do not always
gain conceptual understanding only through numerical presentation. Although many who
teach statistics understand the processes underlying a sequence of formulas and gain con-
ceptual understanding through these formulas, many students do not. Instead, such stu-
dents often need a thorough conceptual explanation to gain such understanding, after which
a numerical presentation may make more sense. Unfortunately, many multiple regression
textbooks assume that students will understand multiple regression best by learning matrix
algebra, wading through formulas, and focusing on details.
At the same time, methods such as structural equation modeling (SEM) and confirma-
tory factor analysis (CFA) are easily taught as extensions of multiple regression. If structured
properly, multiple regression flows naturally into these more complex topics, with nearly
complete carry-over of concepts. Path models (simple SEMs) illustrate and help deal with
some of the problems of MR, CFA does the same for path analysis, and latent variable SEM
combines all the previous topics into a powerful, flexible methodology.
vii
viii • PrefaCe
I have taught courses including these topics at four universities (the University of Iowa,
Virginia Polytechnic Institute & State University, Alfred University, and the University of
Texas). These courses included students and faculty in architecture, engineering, educa-
tional psychology, educational research and statistics, kinesiology, management, political
science, psychology, social work, and sociology, among others. This experience leads me to
believe that it is possible to teach these methods by focusing on the concepts and purposes
of MR and related methods, rather than the derivation and calculation of formulas. Non-
quantitatively-oriented students generally find such an approach clearer, more conceptual,
and less threatening than other approaches. As a result of this conceptual approach, students
become interested in conducting research using MR, CFA, or SEM and are more likely to use
the methods wisely.
THE ORIENTATION OF THIS BOOK
My overriding bias in this book is that these complex methods can be presented and learned
in a conceptual, yet rigorous, manner. I recognize that not all topics are covered in the depth
or detail presented in other texts, but I will direct you to other sources for topics for which
you may want additional detail. My style is also fairly informal; I’ve written this book as if I
were teaching a class.
Data
I also believe that one learns these methods best by doing, and the more interesting and
relevant that “doing,” the better. For this reason, there are numerous example analyses
throughout this book that I encourage you to reproduce as you read. To make this task
easier, the Web site that accompanies the book (www.tzkeith.com) includes the data in a
form that can be used in common statistical analysis programs. Many of the examples are
taken from actual research in the social sciences, and I’ve tried to sample from research
from a variety of areas. In most cases simulated data are provided that mimic the actual
data used in the research. You can reproduce the analyses of the original researchers and,
perhaps, improve on them.
And the data feast doesn’t end there! The Web site also includes data from a major fed-
eral data set: 1000 cases from the National Education Longitudinal Study (NELS) from the
National Center for Education Statistics. NELS was a nationally representative sample of
8th-grade students first surveyed in 1988 and resurveyed in 10th and 12th grades and then
twice after leaving high school. The students’ parents, teachers, and school administrators
were also surveyed. The Web site includes student and parent data from the base year (8th
grade) and student data from the first follow-up (10th grade). Don’t be led astray by the
word Education in NELS; the students were asked an incredible variety of questions, from
drug use to psychological well-being to plans for the future. Anyone with an interest in youth
will find something interesting in these data. Appendix A includes more information about
the data at www.tzkeith.com.
Computer Analysis
Finally, I firmly believe that any book on statistics or research methods should be closely
related to statistical analysis software. Why plug numbers into formulas and churn out
the answers on a calculator—when a statistical program can do the calculations more
quickly and accurately with, for most people, no loss of understanding? Freed from the
need for hand calculations, you can then concentrate on asking and answering important
PrefaCe • ix
research questions, rather than on the intricacies of calculating statistics. This bias toward
computer calculations is especially important for the methods covered in this book, which
quickly become unmanageable by hand. Use a statistical analysis program as you read this
book; do the examples with me and the problems at the end of the chapters, using that
program.
Which program? I use SPSS as my general statistical analysis program, and you can get the
program for a reasonable price as a student in a university (approximately $100 per year for
the “Grad Pack” as this is written). But you need not use SPSS; any of the common packages
will do (e.g., SAS or Stata or R). The output in the text has a generic look to it, which should
be easily translatable to any major statistical package output. In addition, the website (www.
tzkeith.com) includes sample multiple regression and SEM output from various statistical
packages.
For the second half of the book, you will need access to a structural equation modeling
program. Fortunately, student versions of many such programs are available online. Stu-
dent pricing for the program used extensively in this book, Amos, is available, at this writ-
ing, for approximately $50 per year as an SPSS for Windows add-on. Although programs
(and pricing) change, one current limitation of Amos is that there is no Mac OS version
of Amos. If you want to use Amos, you need to be able to run Windows. Amos is, in my
opinion, the easiest SEM program to use (and it produces really nifty pictures). The other
SEM program that I will frequently reference is Mplus. We’ll talk more about SEM in Part
2 of this book. The website for this text has many examples of SEM input and output using
Amos and Mplus.
Overview of the Book
This book is divided into two parts. Part 1 focuses on multiple regression analysis. We begin
by focusing on simple, bivariate regression and then expand that focus into multiple regres-
sion with two, three, and four independent variables. We will concentrate on the analysis
and interpretation of multiple regression as a way of answering interesting and important
research questions. Along the way, we will also deal with the analytic details of multiple
regression so that you understand what is going on when we do a multiple regression analysis.
We will focus on three different types, or flavors, of multiple regression that you will encoun-
ter in the research literature, their strengths and weaknesses, and their proper interpretation.
Our next step will be to add categorical independent variables to our multiple regression
analyses, at which point the relation of multiple regression and ANOVA will become clearer.
We will learn how to test for interactions and curves in the regression line and to apply these
methods to interesting research questions.
The penultimate chapter for Part 1 is a review chapter that summarizes and integrates
what we have learned about multiple regression. Besides serving as a review for those who
have gone through Part 1, it also serves as a useful introduction for those who are interested
primarily in the material in Part 2. In addition, this chapter introduces several important
topics not covered completely in previous chapters. The final chapter in Part 1 presents two
related methods, logistic regression and multilevel modeling, in a conceptual fashion using
what we have learned about multiple regression.
Part 2 focuses on structural equation modeling—the “Beyond” portion of the book’s
title. We begin by discussing path analysis, or structural equation modeling with measured
variables. Simple path analyses are easily estimated via multiple regression analysis, and
many of our questions about the proper use and interpretation of multiple regression will
be answered with this heuristic aid. We will deal in some depth with the problem of valid
versus invalid inferences of causality in these chapters. The problem of error (“the scourge of