Quasi-Experimentation: A Guide to Design and Analysis

382 Pages·2019·3.891 MB·English

by  Charles S. Reichardt

Quasi-Experimentation
Methodology in the Social Sciences
David A. Kenny, Founding Editor
Todd D. Little, Series Editor
www.guilford.com/MSS
This series provides applied researchers and students with analysis and research design books that
emphasize the use of methods to answer research questions. Rather than emphasizing statistical
theory, each volume in the series illustrates when a technique should (and should not) be used and
how the output from available software programs should (and should not) be interpreted. Common
pitfalls as well as areas of further development are clearly articulated.
RECENT VOLUMES
CONFIRMATORY FACTOR ANALYSIS FOR APPLIED RESEARCH,
Second Edition
Timothy A. Brown
PRINCIPLES AND PRACTICE OF STRUCTURAL EQUATION MODELING,
Fourth Edition
Rex B. Kline
HYPOTHESIS TESTING AND MODEL SELECTION IN THE SOCIAL SCIENCES
David L. Weakliem
REGRESSION ANALYSIS AND LINEAR MODELS: Concepts, Applications,
and Implementation
Richard B. Darlington and Andrew F. Hayes
GROWTH MODELING: Structural Equation and Multilevel Modeling Approaches
Kevin J. Grimm, Nilam Ram, and Ryne Estabrook
PSYCHOMETRIC METHODS: Theory into Practice
Larry R. Price
INTRODUCTION TO MEDIATION, MODERATION, AND CONDITIONAL
PROCESS ANALYSIS: A Regression-Based Approach, Second Edition
Andrew F. Hayes
MEASUREMENT THEORY AND APPLICATIONS FOR THE SOCIAL SCIENCES
Deborah L. Bandalos
CONDUCTING PERSONAL NETWORK RESEARCH: A Practical Guide
Christopher McCarty, Miranda J. Lubbers, Raffaele Vacca, and José Luis Molina
QUASI-EXPERIMENTATION: A Guide to Design and Analysis
Charles S. Reichardt
Quasi-Experimentation
A Guide to Design and Analysis
Charles S. Reichardt
Series Editor’s Note by Todd D. Little
THE GUILFORD PRESS
New York London
Copyright © 2019 The Guilford Press
A Division of Guilford Publications, Inc.
370 Seventh Avenue, Suite 1200, New York, NY 10001
www.guilford.com
All rights reserved
No part of this book may be reproduced, translated, stored in a retrieval system,
or transmitted, in any form or by any means, electronic, mechanical, photocopying,
microfilming, recording, or otherwise, without written permission from the publisher.
Printed in the United States of America
This book is printed on acid-free paper.
Last digit is print number: 9 8 7 6 5 4 3 2 1
Library of Congress Cataloging-in-Publication Data
Names: Reichardt, Charles S., author.
Title: Quasi-experimentation : a guide to design and analysis /
Charles S. Reichardt.
Description: New York : Guilford Press, [2019] | Series: Methodology in the
social sciences | Includes bibliographical references and index.
Identifiers: LCCN 2019017566| ISBN 9781462540204 (pbk.) |
ISBN 9781462540259 (hardcover)
Subjects: LCSH: Social sciences—Experiments. | Social sciences—Methodology.
| Experimental design.
Classification: LCC H62 .R4145 2019 | DDC 001.4/34—dc23
LC record available at https://lccn.loc.gov/2019017566
For Stefan, Grace, and Anne

Series Editor’s Note
Research is all about drawing valid conclusions that inform policy and practice. The
randomized clinical trial (RCT) has evolved as the gold standard for drawing causal
inferences but it really isn’t the golden chariot of valid inference. It’s not fool’s gold
either—it’s a sound design; but, thankfully, researchers do have other options, and
sometimes these other options are better suited for a specific research question, par-
ticularly in field settings. Chip Reichardt brings you the wonderful world of valid and
useful designs that, when properly implemented, provide accurate findings. His book
is a delightful guide to the fundamental logic in this other world of inferential research
designs—the quasi- experimental world.
As Reichardt indicates, the distinction between experimental and nonexperimen-
tal or quasi- experimental is more in the thoughtfulness with which the designs are
implemented and in the proper application of the analytics that each design requires.
Even RCTs can yield improper conclusions when they are degraded by factors such as
selective attrition, local treatment effects, treatment noncompliance, variable treatment
fidelity, and the like, particularly when implemented in field settings such as schools,
clinics, and communities. Reichardt brings a thoughtful and practical discussion of all
the issues you need to consider to demonstrate as best as possible the counterfactual
that is a hallmark of accurate inference.
I like the word verisimilitude—the truthlike value of a study’s results. When you
take Reichardt’s advice and implement his tips, your research will benefit by having the
greatest extent of verisimilitude. In this delightfully penned book, Reichardt shares his
vast state-of-the craft understanding for valid conclusions using all manner of inferen-
tial design. Theoretical approaches to inferential designs have matured considerably,
particularly when modern missing- data treatments and best- practice statistical meth-
ods are employed. Having studied and written extensively on these designs, Reichardt
vii
viii Series Editor’s Note
is at the top of the mountain when it comes to understanding and sharing his insights
on these matters. But he does it so effortlessly and accessibly. This book is the kind you
could incorporate into undergraduate curricula where a second course in design and
statistics might be offered. For sure it is a “must” at the graduate level, and even sea-
soned researchers would benefit from the modernization that Reichardt brings to the
inferential designs he covers.
Beyond the thoughtful insights, tips, and wisdom that Reichardt brings to the
designs, his book is extra rich with pedagogical features. He is a very gifted educator
and expertly guides you through the numbered equations using clear and simple lan-
guage. He does the same when he guides you through the output from analysis derived
from each of the designs he covers. Putting accessible words to numbers and core con-
cepts is one of his super powers, which you will see throughout the book as well as in
the glossary of key terms and ideas he compiled. His many and varied examples are
engaging because they span many disciplines. They provide a comprehensive ground-
ing in how the designs can be tailored to address critical questions with which we all
can resonate.
Given that the type of research Reichardt covers here is fundamentally about social
justice (identifying treatment effects as accurately as possible), if we follow his lead, our
findings will change policy and practice to ultimately improve people’s lives. Reichardt
has given us this gift; I ask that you pay it forward by following his lead in the research
you conduct. You will find his sage advice and guidance invaluable. As Reichardt says in
the Preface, “Without knowing the varying effects of treatments, we cannot well know
if our theories of behavior are correct or how to intervene to improve the human condi-
tion.” As always, enjoy!
Todd d. LiTTLe
Society for Research in Child Development meeting
Baltimore, Maryland
Preface
Questions about cause and effect are ubiquitous. For example, we often ask questions
such as the following: How effective is a new diet and exercise program? How likely is it
that an innovative medical regimen will cure cancer? How much does an intensive man-
power training program improve the prospects of the unemployed? How do such effects
vary across different people, settings, times, and outcome measures? Without knowing
the varying effects of treatments, we cannot well know if our theories of behavior are
correct or how to intervene to improve the human condition. Quasi- experiments are
designs frequently used to estimate such effects, and this book will show you how to
use them for that purpose.
This volume explains the logic of both the design of quasi- experiments and the
analysis of the data they produce to provide estimates of treatment effects that are
as credible as can be obtained given the demanding constraints of research practice.
Readers gain both a broad overview of quasi- experimentation and in-depth treatment
of the details of design and analysis. The book brings together the insights of others that
are widely scattered throughout the literature— along with a few insights of my own.
Design and statistical techniques for a full coverage of quasi- experimentation are col-
lected in an accessible format, in a single volume, for the first time.
Although the use of quasi- experiments to estimate the effects of treatments can be
highly quantitative and statistical, you will need only a basic understanding of research
methods and statistical inference, up through multiple regression, to understand the
topics covered in this book. Even then, elementary statistical and methodological top-
ics are reviewed when it would be helpful. All told, the book’s presentation relies on
common sense and intuition far more than on mathematical machinations. As a result,
this book will make the material easier to understand than if you read the original
literature on your own. My purpose is to illuminate the conceptual foundation of
ix