Evaluation Fundamentals PDF
Preview Evaluation Fundamentals
2 This book is dedicated to the ones I love: John C. Beck and, of course, Ingvard. 3 4 Copyright © 2015 by SAGE Publications, Inc. All rights reserved. No part of this book may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system, without permission in writing from the publisher. Printed in the United States of America Library of Congress Cataloging-in-Publication Data Fink, Arlene. Evaluation fundamentals : insights into program effectiveness, quality, and value / Arlene Fink, University of California at Los Angeles, the Langley Research Institute. — Third edition. pages cm Includes bibliographical references and index. ISBN 978-1-4522-8200-8 (pbk. : alk. paper) — ISBN 978-1-4833-1283-5 (web pdf) 1. Medical care—Evaluation. 2. Public health—Evaluation. 3. Medical care—Quality control. 4. Health planning. 5. Outcome assessment (Medical care) I. Title. RA399.A1F563 2015 362.1—dc23 2013038069 This book is printed on acid-free paper. 14 15 16 17 18 10 9 8 7 6 5 4 3 2 1 5 FOR INFORMATION: SAGE Publications, Inc. 2455 Teller Road Thousand Oaks, California 91320 E-mail: [email protected] SAGE Publications Ltd. 1 Oliver’s Yard 55 City Road London EC1Y 1SP United Kingdom SAGE Publications India Pvt. Ltd. B 1/I 1 Mohan Cooperative Industrial Area Mathura Road, New Delhi 110 044 India SAGE Publications Asia-Pacific Pte. Ltd. 3 Church Street #10-04 Samsung Hub Singapore 049483 Acquisitions Editor: Vicki Knight Assistant Editor: Katie Guarino Editorial Assistant: Jessica Miller Production Editor: Stephanie Palermini Copy Editor: Janet Ford Typesetter: C&M Digitals (P) Ltd. Proofreader: Susan Schon Indexer: Sylvia Coates Cover Designer: Anupama Krishnan Marketing Manager: Nicole Elliott 6 Brief Contents Preface About the Author 1. Program Evaluation: A Prelude 2. Evaluation Questions and Evidence of Merit 3. Designing Program Evaluations 4. Sampling 5. Collecting Information: The Right Data Sources 6. Evaluation Measures 7. Managing Evaluation Data 8. Analyzing Evaluation Data 9. Evaluation Reports Answers to Exercises Index 7 Detailed Contents Preface What Is New in the Third Edition About the Author 1. Program Evaluation: A Prelude A Reader’s Guide to Chapter 1 What Is Program Evaluation? The Program or Intervention Program Objectives and Outcomes Program Characteristics Program Impact Program Costs Program Quality Program Value Evaluation Methods Evaluation Questions and Hypotheses Evidence of Merit: Effectiveness, Quality, Value Designing the Evaluation Selecting Participants for the Evaluation Collecting Data on Program Merit Managing Data So That It Can Be Analyzed Analyzing Data to Decide on Program Merit Reporting on Effectiveness, Quality, and Value Who Uses Evaluations? Baseline Data, Formative Evaluation, and Process Evaluation Baseline Data Interim Data and Formative Evaluation Process or Implementation Evaluation Summative Evaluation Qualitative Evaluation Mixed-Methods Evaluation Participatory and Community-Based Evaluation Evaluation Frameworks and Models The PRECEDE-PROCEED Framework RE-AIM The Centers for Disease Control’s Framework for Planning and Implementing Practical Program Evaluation Logic Models Right-to-Left Logic Model Left-to-Right Logic Model 8 Evaluation Reports Online Summary and Transition to the Next Chapter on Evaluation Questions and Evidence of Program Merit Exercises References and Suggested Readings Suggested Websites 2. Evaluation Questions and Evidence of Merit A Reader’s Guide to Chapter 2 Evaluation Questions and Hypotheses Evaluation Questions: Program Goals and Objectives Evaluation Questions: Participants Evaluation Questions: Program Characteristics Evaluation Questions: Financial Costs Evaluation Questions: The Program’s Environment Evidence of Merit Sources of Evidence Evidence by Comparison Evidence From Expert Consultation: Professionals, Consumers, Community Groups Evidence From Existing Data and Large Databases (“Big Data”) Evidence From the Research Literature When to Decide on Evidence Program Evaluation and Economics The QEV Report: Questions, Evidence, Variables Summary and Transition to the Next Chapter on Designing Program Evaluations Exercises References and Suggested Readings 3. Designing Program Evaluations A Reader’s Guide to Chapter 3 Evaluation Design: Creating the Structure Experimental Designs The Randomized Controlled Trial or RCT Parallel Controls Wait-List Controls Factorial Designs Randomizing and Blinding Random Assignment Random Clusters Improving on Chance: Stratifying and Blocking Blinding Nonrandomized Controlled Trials Parallel Controls The Problem of Incomparable Participants: Statistical Methods Like ANCOVA to the Rescue Observational Designs Cross-Sectional Designs Cohort Designs 9 Case Control Designs A Note on Pretest-Posttest Only or Self-Controlled Designs Comparative Effectiveness Research and Evaluation Commonly Used Evaluation Designs Internal and External Validity Internal Validity Is Threatened External Validity Is Threatened Summary and Transition to the Next Chapter on Sampling Exercises References and Suggested Readings 4. Sampling A Reader’s Guide to Chapter 4 What Is a Sample? Why Sample? Inclusion and Exclusion Criteria or Eligibility Sampling Methods Simple Random Sampling Random Selection and Random Assignment Systematic Sampling Stratified Sampling Cluster Sampling Nonprobability or Convenience Sampling The Sampling Unit Sample Size Power Analysis and Alpha and Beta Errors The Sampling Report Summary and Transition to the Next Chapter on Collecting Information Exercises References and Suggested Readings Suggested Websites 5. Collecting Information: The Right Data Sources A Reader’s Guide to Chapter 5 Information Sources: What’s the Question? Choosing Appropriate Data Sources Data Sources or Measures in Program Evaluation and Their Advantages and Disadvantages Self-Administered Surveys Achievement Tests Record Reviews Observations Interviews Computer-Assisted Interviews Physical Examinations Large Databases Vignettes The Literature 10
The list of books you might like
