ebook img

Essentials of Utilization-Focused Evaluation PDF

489 Pages·2011·7.209 MB·English
Save to my drive
Quick download
Download
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview Essentials of Utilization-Focused Evaluation

of ESSENTIALS UTILIZATION-FOCUSED EVALUATION To Jean K. Gornick, my brevity muse. The calculus she holds before me: “Fewer words, more readers, more smiles, greater utility.” of ESSENTIALS UTILIZATION-FOCUSED EVALUATION MICHAEL QUINN PATTON Utilization-Focused Evaluation, Saint Paul, Minnesota (cid:45)(cid:54)(cid:57)(cid:3)(cid:48)(cid:53)(cid:45)(cid:54)(cid:57)(cid:52)(cid:40)(cid:59)(cid:48)(cid:54)(cid:53)(cid:33)(cid:3) Copyright© 2012 by SAGE Publications, Inc. (cid:58)(cid:40)(cid:46)(cid:44)(cid:3)(cid:55)(cid:92)(cid:73)(cid:83)(cid:80)(cid:74)(cid:72)(cid:91)(cid:80)(cid:86)(cid:85)(cid:90)(cid:19)(cid:3)(cid:48)(cid:85)(cid:74)(cid:21) All rights reserved. No part of this book may be (cid:25)(cid:27)(cid:28)(cid:28)(cid:3)(cid:59)(cid:76)(cid:83)(cid:83)(cid:76)(cid:89)(cid:3)(cid:57)(cid:86)(cid:72)(cid:75) reproduced or utilized in any form or by any (cid:59)(cid:79)(cid:86)(cid:92)(cid:90)(cid:72)(cid:85)(cid:75)(cid:3)(cid:54)(cid:72)(cid:82)(cid:90)(cid:19)(cid:3)(cid:42)(cid:72)(cid:83)(cid:80)(cid:77)(cid:86)(cid:89)(cid:85)(cid:80)(cid:72)(cid:3)(cid:32)(cid:24)(cid:26)(cid:25)(cid:23) means, electronic or mechanical, including photocopying, recording, or by any information (cid:44)(cid:20)(cid:84)(cid:72)(cid:80)(cid:83)(cid:33)(cid:3)(cid:86)(cid:89)(cid:75)(cid:76)(cid:89)(cid:39)(cid:90)(cid:72)(cid:78)(cid:76)(cid:87)(cid:92)(cid:73)(cid:21)(cid:74)(cid:86)(cid:84) storage and retrieval system, without permission in writing from the publisher. (cid:58)(cid:40)(cid:46)(cid:44)(cid:3)(cid:55)(cid:92)(cid:73)(cid:83)(cid:80)(cid:74)(cid:72)(cid:91)(cid:80)(cid:86)(cid:85)(cid:90)(cid:3)(cid:51)(cid:91)(cid:75)(cid:21)(cid:3) (cid:24)(cid:3)(cid:54)(cid:83)(cid:80)(cid:93)(cid:76)(cid:89)(cid:187)(cid:90)(cid:3)(cid:64)(cid:72)(cid:89)(cid:75) (cid:28)(cid:28)(cid:3)(cid:42)(cid:80)(cid:91)(cid:96)(cid:3)(cid:57)(cid:86)(cid:72)(cid:75) (cid:51)(cid:86)(cid:85)(cid:75)(cid:86)(cid:85)(cid:3)(cid:44)(cid:42)(cid:24)(cid:64)(cid:3)(cid:24)(cid:58)(cid:55) (cid:60)(cid:85)(cid:80)(cid:91)(cid:76)(cid:75)(cid:3)(cid:50)(cid:80)(cid:85)(cid:78)(cid:75)(cid:86)(cid:84) (cid:58)(cid:40)(cid:46)(cid:44)(cid:3)(cid:55)(cid:92)(cid:73)(cid:83)(cid:80)(cid:74)(cid:72)(cid:91)(cid:80)(cid:86)(cid:85)(cid:90)(cid:3)(cid:48)(cid:85)(cid:75)(cid:80)(cid:72)(cid:3)(cid:55)(cid:93)(cid:91)(cid:21)(cid:3)(cid:51)(cid:91)(cid:75)(cid:21)(cid:3) (cid:41)(cid:3)(cid:24)(cid:22)(cid:48)(cid:3)(cid:24)(cid:3)(cid:52)(cid:86)(cid:79)(cid:72)(cid:85)(cid:3)(cid:42)(cid:86)(cid:86)(cid:87)(cid:76)(cid:89)(cid:72)(cid:91)(cid:80)(cid:93)(cid:76)(cid:3)(cid:48)(cid:85)(cid:75)(cid:92)(cid:90)(cid:91)(cid:89)(cid:80)(cid:72)(cid:83)(cid:3)(cid:40)(cid:89)(cid:76)(cid:72) Printed in the United States of America (cid:52)(cid:72)(cid:91)(cid:79)(cid:92)(cid:89)(cid:72)(cid:3)(cid:57)(cid:86)(cid:72)(cid:75)(cid:19)(cid:3)(cid:53)(cid:76)(cid:94)(cid:3)(cid:43)(cid:76)(cid:83)(cid:79)(cid:80)(cid:3)(cid:24)(cid:24)(cid:23)(cid:3)(cid:23)(cid:27)(cid:27)(cid:3) Library of Congress Cataloging-in-Publication Data (cid:48)(cid:85)(cid:75)(cid:80)(cid:72) Patton, Michael Quinn. (cid:58)(cid:40)(cid:46)(cid:44)(cid:3)(cid:55)(cid:92)(cid:73)(cid:83)(cid:80)(cid:74)(cid:72)(cid:91)(cid:80)(cid:86)(cid:85)(cid:90)(cid:3)(cid:40)(cid:90)(cid:80)(cid:72)(cid:20)(cid:55)(cid:72)(cid:74)(cid:80)(cid:196)(cid:74)(cid:3)(cid:55)(cid:91)(cid:76)(cid:21)(cid:3)(cid:51)(cid:91)(cid:75)(cid:21) Essentials of utilization-focused evaluation/Michael (cid:26)(cid:26)(cid:3)(cid:55)(cid:76)(cid:82)(cid:80)(cid:85)(cid:3)(cid:58)(cid:91)(cid:89)(cid:76)(cid:76)(cid:91)(cid:3)(cid:10)(cid:23)(cid:25)(cid:20)(cid:23)(cid:24) Quinn Patton. (cid:45)(cid:72)(cid:89)(cid:3)(cid:44)(cid:72)(cid:90)(cid:91)(cid:3)(cid:58)(cid:88)(cid:92)(cid:72)(cid:89)(cid:76) (cid:58)(cid:80)(cid:85)(cid:78)(cid:72)(cid:87)(cid:86)(cid:89)(cid:76)(cid:3)(cid:23)(cid:27)(cid:31)(cid:30)(cid:29)(cid:26)(cid:3) p. cm. Includes bibliographical references and index. ISBN 978-1-4129-7741-8 (pbk.) 1. Evaluation research (Social action programs)— (cid:40)(cid:74)(cid:88)(cid:92)(cid:80)(cid:90)(cid:80)(cid:91)(cid:80)(cid:86)(cid:85)(cid:90)(cid:3)(cid:44)(cid:75)(cid:80)(cid:91)(cid:86)(cid:89)(cid:33)(cid:3) (cid:61)(cid:80)(cid:74)(cid:82)(cid:80)(cid:3)(cid:50)(cid:85)(cid:80)(cid:78)(cid:79)(cid:91) United States. I. Title. (cid:40)(cid:90)(cid:90)(cid:86)(cid:74)(cid:80)(cid:72)(cid:91)(cid:76)(cid:3)(cid:44)(cid:75)(cid:80)(cid:91)(cid:86)(cid:89)(cid:33)(cid:3) (cid:51)(cid:72)(cid:92)(cid:89)(cid:76)(cid:85)(cid:3)(cid:47)(cid:72)(cid:73)(cid:80)(cid:73) H62.5.U5P3688 2012 (cid:44)(cid:75)(cid:80)(cid:91)(cid:86)(cid:89)(cid:80)(cid:72)(cid:83)(cid:3)(cid:40)(cid:90)(cid:90)(cid:80)(cid:90)(cid:91)(cid:72)(cid:85)(cid:91)(cid:33)(cid:3) (cid:50)(cid:72)(cid:83)(cid:80)(cid:76)(cid:3)(cid:50)(cid:86)(cid:90)(cid:74)(cid:80)(cid:76)(cid:83)(cid:72)(cid:82) 300.72—dc23 2011019187 (cid:55)(cid:89)(cid:86)(cid:75)(cid:92)(cid:74)(cid:91)(cid:80)(cid:86)(cid:85)(cid:3)(cid:44)(cid:75)(cid:80)(cid:91)(cid:86)(cid:89)(cid:33)(cid:3) (cid:40)(cid:90)(cid:91)(cid:89)(cid:80)(cid:75)(cid:3)(cid:61)(cid:80)(cid:89)(cid:75)(cid:80)(cid:85)(cid:78) (cid:42)(cid:86)(cid:87)(cid:96)(cid:3)(cid:44)(cid:75)(cid:80)(cid:91)(cid:86)(cid:89)(cid:33)(cid:3) (cid:59)(cid:76)(cid:89)(cid:89)(cid:80)(cid:3)(cid:51)(cid:76)(cid:76)(cid:3)(cid:55)(cid:72)(cid:92)(cid:83)(cid:90)(cid:76)(cid:85) (cid:59)(cid:96)(cid:87)(cid:76)(cid:90)(cid:76)(cid:91)(cid:91)(cid:76)(cid:89)(cid:33)(cid:3) (cid:42)(cid:13)(cid:52)(cid:3)(cid:43)(cid:80)(cid:78)(cid:80)(cid:91)(cid:72)(cid:83)(cid:90)(cid:3)(cid:15)(cid:55)(cid:16)(cid:3)(cid:51)(cid:91)(cid:75)(cid:21) (cid:55)(cid:89)(cid:86)(cid:86)(cid:77)(cid:89)(cid:76)(cid:72)(cid:75)(cid:76)(cid:89)(cid:33)(cid:3) (cid:43)(cid:76)(cid:85)(cid:85)(cid:80)(cid:90)(cid:3)(cid:62)(cid:21)(cid:3)(cid:62)(cid:76)(cid:73)(cid:73) (cid:48)(cid:85)(cid:75)(cid:76)(cid:95)(cid:76)(cid:89)(cid:33)(cid:3) (cid:50)(cid:72)(cid:91)(cid:79)(cid:83)(cid:76)(cid:76)(cid:85)(cid:3)(cid:55)(cid:72)(cid:87)(cid:72)(cid:89)(cid:74)(cid:79)(cid:86)(cid:85)(cid:91)(cid:80)(cid:90) (cid:42)(cid:86)(cid:93)(cid:76)(cid:89)(cid:3)(cid:43)(cid:76)(cid:90)(cid:80)(cid:78)(cid:85)(cid:76)(cid:89)(cid:33)(cid:3) (cid:42)(cid:72)(cid:85)(cid:75)(cid:80)(cid:74)(cid:76)(cid:3)(cid:47)(cid:72)(cid:89)(cid:84)(cid:72)(cid:85) This book is printed on acid-free paper. (cid:52)(cid:72)(cid:89)(cid:82)(cid:76)(cid:91)(cid:80)(cid:85)(cid:78)(cid:3)(cid:52)(cid:72)(cid:85)(cid:72)(cid:78)(cid:76)(cid:89)(cid:33)(cid:3) (cid:47)(cid:76)(cid:83)(cid:76)(cid:85)(cid:3)(cid:58)(cid:72)(cid:83)(cid:84)(cid:86)(cid:85) (cid:55)(cid:76)(cid:89)(cid:84)(cid:80)(cid:90)(cid:90)(cid:80)(cid:86)(cid:85)(cid:90)(cid:3)(cid:44)(cid:75)(cid:80)(cid:91)(cid:86)(cid:89)(cid:33)(cid:3) (cid:40)(cid:75)(cid:76)(cid:83)(cid:76)(cid:3)(cid:47)(cid:92)(cid:91)(cid:74)(cid:79)(cid:80)(cid:85)(cid:90)(cid:86)(cid:85) 11 12 13 14 15 10 9 8 7 6 5 4 3 2 1 Contents Preface xvii About the Author xxi Introduction, Overview, and Context 1 Utilization-Focused Reality Testing: Finding Out if What Is Hoped for Actually Happens Step 1. Assess and Build Program and Organizational Readiness for Utilization-Focused Evaluation 15 Step 2. Assess and Enhance Evaluator Readiness and Competence to Undertake a Utilization-Focused Evaluation 37 Interlude, Steps 1 and 2. Complex Dynamic Systems Interconnections: Assessing the Match Between the Program and Evaluator, the Interrelationship Between Steps 1 and 2 59 Step 3. Identify, Organize, and Engage Primary Intended Users: The Personal Factor 61 Interlude, Steps 1, 2, and 3. Complex Dynamic Systems Interconnections: Assessing the Readiness of Primary Intended Users to Engage 85 Step 4. Situation Analysis Conducted Jointly With Primary Intended Users 87 Interlude, Steps 1 Through 4. Complex Dynamic Systems Interconnections: Assessing the Alignment Among the Steps and Integrating the Situation Analysis From Steps 1 Through 4 111 Step 5. Identify and Prioritize Primary Intended Uses by Determining Priority Purposes 113 Interlude, Steps 1 Through 5. Complex Dynamic Systems Interconnections: Focusing on Intended Uses by Intended Users 138 Step 6. Consider and Build in Process Uses if and as Appropriate 140 Interlude, Steps 1 Through 6. Complex Dynamic Systems Interconnections: Integrating Findings Uses and Process Uses 166 Step 7. Focus Priority Evaluation Questions 169 Interlude, Steps 1 Through 7. Complex Dynamic Systems Interconnections: Questioning and Focusing as Part of Each Step Throughout the U-FE Process 188 Step 8. Check That Fundamental Areas for Evaluation Inquiry Are Being Adequately Addressed: Implementation, Outcomes, and Attribution Questions 191 Interlude, Steps 1 Through 8. Complex Dynamic Systems Interconnections: Attending to Fundamental Interconnections 228 Step 9. Determine What Intervention Model or Theory of Change Is Being Evaluated 230 Interlude, Steps 1 Through 9. Complex Dynamic Systems Interconnections: Considering Model and Theory Formulation Issues Throughout the U-FE Process 261 Step 10. Negotiate Appropriate Methods to Generate Credible Findings That Support Intended Use by Intended Users 263 Interlude, Steps 1 Through 10. Complex Dynamic Systems Interconnections: Attending to Methods and Design Decisions Throughout the U-FE Process 282 Step 11. Make Sure Intended Users Understand Potential Methods Controversies and Their Implications 284 Interlude, Steps 1 Through 11. Complex Dynamic Systems Interconnections: Making Methods Choices That Are Appropriate 307 Step 12. Simulate Use of Findings: Evaluation’s Equivalent of a Dress Rehearsal 309 Step 13. Gather Data With Ongoing Attention to Use 323 Step 14. Organize and Present the Data for Interpretation and Use by Primary Intended Users: Analysis, Interpretation, Judgment, and Recommendations 336 Step 15. Prepare an Evaluation Report to Facilitate Use and Disseminate Significant Findings to Expand Influence 365 Step 16. Follow Up With Primary Intended Users to Facilitate and Enhance Use 380 Interlude, Interconnections Among Steps 14–16. U-FE as a Complex Dynamic System: Facilitating Use 386 Step 17. Metaevaluation of Use: Be Accountable, Learn, and Improve 388 Summary and Conclusion 403 Utilization-Focused Evaluation Checklist: 17 Steps to Evaluations That Are Useful—and Actually Used 406 U-FE Complex Dynamic and Adaptive Systems Graphic: Interactions Among All 17 Steps 425 References 427 Author Index 441 Subject Index 445 Detailed Contents Preface xvii About the Author xxi Introduction, Overview, and Context 1 Utilization-Focused Reality Testing: Finding Out if What Is Hoped for Actually Happens 1. Step 1: Assess and Build Program and Organizational Readiness for Utilization-Focused Evaluation 15 Learning About and Assessing the Evaluation Context 18 Formally Launching the Evaluation: Exercises to Help Assess and Build Readiness for Evaluation—and Get the Process Started 19 Initiating a Utilization-Focused Evaluation 35 Practice Exercises 36 2. Step 2: Assess and Enhance Evaluator Readiness and Competence to Undertake a Utilization-Focused Evaluation 37 Essential Competencies 38 Evaluator Credibility 40 Utilization-Focused Evaluation Competence and Credibility 42 Technical and Methodological Competence: Utilization-Focused Pragmatism 44 Evaluation Negotiations 44 Being Active-Reactive-Interactive-Adaptive 47 Adaptive Management 47 Multiple Evaluator Roles and Individual Style 49 Evaluation Teams 50 Cultural Competence 50 Respect for Intended Users 53 Balancing Attention to Tasks and Relationships in Group Facilitation 53 Reflexivity 55 Self-Knowledge: Beyond Narcissism 56 Practice Exercises 57 Interlude, Steps 1 and 2. Complex Dynamic Systems Interconnections: Assessing the Match Between the Program and Evaluator, the Interrelationship Between Steps 1 and 2 59 3. Step 3: Identify, Organize, and Engage Primary Intended Users: The Personal Factor 61 Asking Users About Use 61 The Personal Factor 62 Data on the Importance of the Personal Factor 63 Supporting Research on the Personal Factor 65 Identifying and Working With Primary Intended Users 66 Intended Users at Different Levels: From Local to National and International Engagement 67 Evaluation Use Exemplars 69 Evaluation’s Premier Lesson 71 Practical Implications of the Personal Factor: Selecting and Working With Primary Intended Users 72 Temptations Away From Engaging Primary Intended Users 79 Practice Exercises 84 Interlude, Steps 1, 2, and 3. Complex Dynamic Systems Interconnections: Assessing the Readiness of Primary Intended Users to Engage 85 4. Step 4: Situation Analysis Conducted Jointly With Primary Intended Users 87 Context Matters: The Challenge of Situation Analysis in Designing and Conducting Evaluations 88 Force Field Analysis 90 Identifying Factors and Forces 90 Internal and External Evaluators 96 Themes From Internal Evaluators 98 Internal Role Definitions 99 Internal-External Evaluation Combinations 100 Evaluation as a Leadership Function 102 Evaluation Team Composition Analysis 104 Balancing Task and Relationship Demands 105 Risk Assessment and Contingency Thinking 106 Three Types of Risk 107

See more

The list of books you might like

Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.