Table Of Content2
This book is dedicated to the ones I love: John C. Beck and, of course, Ingvard.
3
4
Copyright © 2015 by SAGE Publications, Inc.
All rights reserved. No part of this book may be reproduced or utilized in any form or by any means, electronic or mechanical, including
photocopying, recording, or by any information storage and retrieval system, without permission in writing from the publisher.
Printed in the United States of America
Library of Congress Cataloging-in-Publication Data
Fink, Arlene.
Evaluation fundamentals : insights into program effectiveness, quality, and value / Arlene Fink, University of California at Los Angeles, the
Langley Research Institute. — Third edition.
pages cm
Includes bibliographical references and index.
ISBN 978-1-4522-8200-8 (pbk. : alk. paper) —
ISBN 978-1-4833-1283-5 (web pdf)
1. Medical care—Evaluation. 2. Public health—Evaluation.
3. Medical care—Quality control. 4. Health planning.
5. Outcome assessment (Medical care) I. Title.
RA399.A1F563 2015
362.1—dc23 2013038069
This book is printed on acid-free paper.
14 15 16 17 18 10 9 8 7 6 5 4 3 2 1
5
FOR INFORMATION:
SAGE Publications, Inc.
2455 Teller Road
Thousand Oaks, California 91320
E-mail: order@sagepub.com
SAGE Publications Ltd.
1 Oliver’s Yard
55 City Road
London EC1Y 1SP
United Kingdom
SAGE Publications India Pvt. Ltd.
B 1/I 1 Mohan Cooperative Industrial Area
Mathura Road, New Delhi 110 044
India
SAGE Publications Asia-Pacific Pte. Ltd.
3 Church Street
#10-04 Samsung Hub
Singapore 049483
Acquisitions Editor: Vicki Knight
Assistant Editor: Katie Guarino
Editorial Assistant: Jessica Miller
Production Editor: Stephanie Palermini
Copy Editor: Janet Ford
Typesetter: C&M Digitals (P) Ltd.
Proofreader: Susan Schon
Indexer: Sylvia Coates
Cover Designer: Anupama Krishnan
Marketing Manager: Nicole Elliott
6
Brief Contents
Preface
About the Author
1. Program Evaluation: A Prelude
2. Evaluation Questions and Evidence of Merit
3. Designing Program Evaluations
4. Sampling
5. Collecting Information: The Right Data Sources
6. Evaluation Measures
7. Managing Evaluation Data
8. Analyzing Evaluation Data
9. Evaluation Reports
Answers to Exercises
Index
7
Detailed Contents
Preface
What Is New in the Third Edition
About the Author
1. Program Evaluation: A Prelude
A Reader’s Guide to Chapter 1
What Is Program Evaluation?
The Program or Intervention
Program Objectives and Outcomes
Program Characteristics
Program Impact
Program Costs
Program Quality
Program Value
Evaluation Methods
Evaluation Questions and Hypotheses
Evidence of Merit: Effectiveness, Quality, Value
Designing the Evaluation
Selecting Participants for the Evaluation
Collecting Data on Program Merit
Managing Data So That It Can Be Analyzed
Analyzing Data to Decide on Program Merit
Reporting on Effectiveness, Quality, and Value
Who Uses Evaluations?
Baseline Data, Formative Evaluation, and Process Evaluation
Baseline Data
Interim Data and Formative Evaluation
Process or Implementation Evaluation
Summative Evaluation
Qualitative Evaluation
Mixed-Methods Evaluation
Participatory and Community-Based Evaluation
Evaluation Frameworks and Models
The PRECEDE-PROCEED Framework
RE-AIM
The Centers for Disease Control’s Framework for Planning and Implementing Practical Program Evaluation
Logic Models
Right-to-Left Logic Model
Left-to-Right Logic Model
8
Evaluation Reports Online
Summary and Transition to the Next Chapter on Evaluation Questions and Evidence of Program Merit
Exercises
References and Suggested Readings
Suggested Websites
2. Evaluation Questions and Evidence of Merit
A Reader’s Guide to Chapter 2
Evaluation Questions and Hypotheses
Evaluation Questions: Program Goals and Objectives
Evaluation Questions: Participants
Evaluation Questions: Program Characteristics
Evaluation Questions: Financial Costs
Evaluation Questions: The Program’s Environment
Evidence of Merit
Sources of Evidence
Evidence by Comparison
Evidence From Expert Consultation: Professionals, Consumers, Community Groups
Evidence From Existing Data and Large Databases (“Big Data”)
Evidence From the Research Literature
When to Decide on Evidence
Program Evaluation and Economics
The QEV Report: Questions, Evidence, Variables
Summary and Transition to the Next Chapter on Designing Program Evaluations
Exercises
References and Suggested Readings
3. Designing Program Evaluations
A Reader’s Guide to Chapter 3
Evaluation Design: Creating the Structure
Experimental Designs
The Randomized Controlled Trial or RCT
Parallel Controls
Wait-List Controls
Factorial Designs
Randomizing and Blinding
Random Assignment
Random Clusters
Improving on Chance: Stratifying and Blocking
Blinding
Nonrandomized Controlled Trials
Parallel Controls
The Problem of Incomparable Participants: Statistical Methods Like ANCOVA to the Rescue
Observational Designs
Cross-Sectional Designs
Cohort Designs
9
Case Control Designs
A Note on Pretest-Posttest Only or Self-Controlled Designs
Comparative Effectiveness Research and Evaluation
Commonly Used Evaluation Designs
Internal and External Validity
Internal Validity Is Threatened
External Validity Is Threatened
Summary and Transition to the Next Chapter on Sampling
Exercises
References and Suggested Readings
4. Sampling
A Reader’s Guide to Chapter 4
What Is a Sample?
Why Sample?
Inclusion and Exclusion Criteria or Eligibility
Sampling Methods
Simple Random Sampling
Random Selection and Random Assignment
Systematic Sampling
Stratified Sampling
Cluster Sampling
Nonprobability or Convenience Sampling
The Sampling Unit
Sample Size
Power Analysis and Alpha and Beta Errors
The Sampling Report
Summary and Transition to the Next Chapter on Collecting Information
Exercises
References and Suggested Readings
Suggested Websites
5. Collecting Information: The Right Data Sources
A Reader’s Guide to Chapter 5
Information Sources: What’s the Question?
Choosing Appropriate Data Sources
Data Sources or Measures in Program Evaluation and Their Advantages and Disadvantages
Self-Administered Surveys
Achievement Tests
Record Reviews
Observations
Interviews
Computer-Assisted Interviews
Physical Examinations
Large Databases
Vignettes
The Literature
10
