ERIC ED343948: Tracking Your School's Success: A Guide to Sensible Evaluation.

162 Pages·1992·3.7 MB·English

by  ERIC

DOCUMENT RESUME
TM 018 131
ED 343 948
Lynn
Herman, Joan L.; Winterv,,
AUTHOR
Sensible
Success: A Guide to
Tracking Your School's
TITLE
Evaluation.
Evaluation, Standards, and
Center for Research on
INSTITUTION
Angeles, CA.
Student Testing, Los
(ED),
Research and Improvement
Office of Educational
SPONS AGENCY
Washington, DC.
ISBN-0-8039-6024-7
REPORT NO
92
PUB DATE
G00869003
CONTRACT
of "Sensible School-Based
380p.; Revised version
NOTE
Evaluation", see ED 329 575.
Teller Road, Newbury Park,
Corwin Press, Inc., 2455
AVAILABLE FROM
CA 91320 ($21.95).
(055) -- Tests/Evaluation
Guides - Non-Classroom Use
PUB TYPE
Instruments (160)
MF01/PC08 Plus Postage.
EDRS PRICE
Making; *Educational
Data Analysis; *Decision
DESCRIPTORS
Change; Elementary Secondary
Assessment; *Educational
Methods; Formative Evaluation;
Education; *Evaluation
Evaluation; Needs
*Guidelines; *Institutional
School Based Management;
Assessment; Productivity;
Success
Progress Reporting
IDENTIFIERS
ABSTRACT
increase a school's
This guide is designed to
techniques for
simple yet powerful
productivity by presenting some
the most
making decisions to address
looking at the school and
and using
success is tracking progress
immediate problems. Tracking
school-based
implementing changes from
evaluation as a tool for
review the
restructuring. Chapters, which
management to complete
(1) "Introduction. Evaluation--A
basic steps of evaluation, are:
Evaluation"; (3) "Step
(2) "Step One: Focus the
Management Tool";
Manage
Strategies"; (4) "Step Three:
Two: Identify Tracking
Scoring
Collection"; (5) "Step Four:
Instrument Development and Data
Information and
(6) "Step Five: Analyze
and Summarizing Data";
Program
and Contin
"Step Six: Use Findings
Interpret Findings"; (7)
contain
and Revelation." Five resources
MoAitoring"; and (8) "Reprise
sample
data collection, including a
sample forms for evaluation
tables and one figure summarize
school report card. Seventeen
Suggested
there are six sample worksheets.
evaluation techniques, and
each chapter. (SLD)
readings (35 in all) follow
***********************************************************************
the best that can be made
Reproductions supplied by EDRS are
from the original document.
************IV*************************************************X******v.*
Tracking
Your
Sch ors
Success:
A Guide to Sensible Evaluation
D Joan L. Herman
0 Lynn Winters
CORWIN PRESS, INC.
A Sage Publications Company
Newbury Park, California
Copyright © 1992 by The Regents of the University of California
All rights reserved. The purchase of this copyrighted material confers the right
on the purchaser to reproduce the "Resource" sections by photocopying without
specific authorization by the copyright holder. No other part of this book may
be reproduced or utilized in any form or by any means, electronic ol mechanical,
including photocopying, recording, or by any information storage and retrieval
system, without permission in writing from the publisher.
This work was developed at the Center for Research on Evaluation,
Standards, and Student Testing, Center for the Study of Evaluation, Grad-
uate School of Education, University of California, Los Angeles.
The research reported herein was conducted with partial support from
the U.S. Department of Education, Office of Educational Research and
Improvement, pursuant to Grant No, G00869003. However, the opinions
expressed do not necessarily reflect the position or policy of this agency, and
no official endorsement by the agency should be inferred.
This work is published and distributed by Corwin Press, Inc., Newbury
Park, California, under an exclusive agreement with The Regents of the
University of California.
For information address:
Corwin Press, Inc.
A Sage Publications Company
2456 Teller Road
Newbury Park, California 91320
SAGE Publications Ltd.
6 Ronhill Street
London EC2A 4PU
United Kingdom
SAGE Publications India Pvt. Ltd.
M-32 Market
Greater Kailash I
New Delhi 110 048 India
Printed in the United States of America
Library of Congress Cataloging-in-Publication Data
Herman, Joan L.
Tracking your school's success: a guide to sensible evaluation /
Joan L. Herman, Lynn Winters.
cm.
p.
) and index.
Includes bibliographical references (p.
ISBN 0-8039-6024-7
1. Educational evaluationUnited States. 2. School management
and organizationUnited StatesDecision-making.
I. Winters,
Lynn. II. Title.
LB2822.75.H47 1992
92-3695
379.1'54dc20
CIP
7 6 5 4 3 2 1
92 93 94 95 10 9
Corwin Press Production Editor: Tara S. Mead
7
Contents
Acknowledgments
viii
About the Authors
Introduction. EvaluationA Management
Tool
1
New Demands on the Schools
1.
Why Evaluate?
2
What Is School-Based Evaluation?
6
What Are the Pitfalls in Evaluation?
8
A Better Approach: School-Based
Evaluation
10
When to Track Your Progress With
Evaluation
12
Involve Others and Increase Your
Power
12
Tracking Your Success in Six Steps
13
How to Use This Manual
14
References
15
Suggested Readings
16
Step One. Focus the Evaluation
17
Focus on Major Program Aspects
18
Develop a Comprehensive View
of School or
Program Quality
18
Choose a Wide-Angle
or Close-Up Perspective
19
Needs Assessment Questions
20
Formative Evaluation Questions
21
Select and Refine Priority Questions
27
References
30
Suggested Readings
30
Strategies
33
Step Two. Identify Tracking
33
What Evidence Are We Looking For?
34
Our Evidence?
Where and How Are We Going to Find
Select Tracking Strategies by Balancing
37
Competing Requirements
Address
Use Multiple Information Sources to
40
Each Question
40
What Standards Will We Use?
43
Suggested Readings
and
Step Three. Manage Instrument Development
47
Data Collection
48
Consolidate Your Evaluation :interests
Create a Blueprint for Instrument Development
49
or Instrument Selection
49
Create a Consolidated Management Plan
57
Avoiding Data CollecCon Disasters
62
Suggested Readings
65
Step Four. Scoring and Summarizing Data
65
What You Should Know Before Scoring
70
Selecting Scoring Procedures
74
Score Summaries and Score Reports
75
Suggested Readings
Findings
79
Step Five. Analyze Information and Interpret
79
Data Interpretation: A Dialogue About Meaning
81
Organize Your Data by Question
86
Answering Needs Assessment Questions
94
Answering Formative Evaluation Questions
98
Suggested Readings
Monitoring
99
Step Six. Use Findings and Continue Program
99
Communicate Findings
106
Develop an Action Plan
108
Continue Program Monitoring
108
Suggested Readings
Reprise and Revelation
111
111
Evaluation Is Much Faster the Second Time Around
114
It's Not Too Late to Automate
117
Getting by With a T.ittle Help From Your Friends
120
Tracking Your Success: The Road to Success
121
Suggested Readings
Resource A: Sample Parent Survey
123
Resource B: Sample Tea( 'her Survey
125
Resource C: Sample Student Survey
130
Resource D: Worksheets
142
Resource E: Sample Report Card
150
Index
155
Acknowledgments
This book is the product of a multi-year research project conducted
Standards, and Student
by the Center for Research on Evaluation,
States Department
Testing (CRESST) with funding from the United
and Improvement, Our
of Education, Office of Education Research
the contributions and
project would not have been possible without
We gratefully
cooperation of many individuals and organizations.
acknowledge their help and support.
Teachers and administrators in schools across the country who
pilot-tPsted the evaluation model.
through-
School board members, superintendents, and principals
research on evalua-
out California who participated in our
tion utility and reporting issues.
Members of the California School Boards Association and the As-
sociation of California School Administrators who provided
valuable review of early versions.
Colleagues in Division H of the American Education Research
Association who critiqued our initial concept and provided
needed encouragement, with particular thanks for support
and suggestions from Carol Robinson, Todd Endo, Sandy
Williams, Stella Port, Mardell Kolls, Tom Mann, and Dick
Jaeger.
Special thanks to Covina Valley School District, Jack Price,
Superintendent, for the use of their model school report cards. At
CRESST/UCLA, we are grateful to Eva Baker, CRESST co-director,
viii
Acknowledgments
for her conceptualization of sen dime evaluation which undergirds the
our work. Our sincere thanks also to Shari Golan who directed the
library research and coordinated the day-to-day operations; and to
Judy Miyoshi and Melissa Goldberg who provided excellent ad-
ministrative and secretarial support.
Finally, we would like to thank the following friends and family
for the intellectual, emotional, and sensible support they provided for
this project:
Michelle Herman, Joel Muhoz, and Karla Winters
About the Authors
Joan L. Herman is Associate Director of the UCLA Center for the
Study of Evaluation and its Center for Research on Evaluation,
Standards, and Student Testing. A former elementary teacher and
current Chair of the Board of Education at Stephen S. Wise School,
her research interests focus on the use of evaluation and testing to
improve schools, the development of multipurpose school-based eval-
uation systems, and the effects of technology in schools. She has been
active in teacher training, recently in the area of developing alterna-
tive assessments of student performance, and is widely experienced
in program evaluation at all levels. She is editor of the popular second
edition of the Program Evaluation Kit and primary author of its
Evaluator's Handbook. Other recent books include Making Schools
Work for Underachieving Minority Students.
Lynn Winters is Assessment Director of the Galef Institute, a non-
profit educational organization dedicated to serving at-risk students
through arts-based curricular and instructional interventions. She
currently is directing the development of performance and portfolio
assessments for a K-6 arts-inf-sed social studies curriculum. She has
more than 15 years experience in school-level evaluation as both a
research director and a consultant since leaving the high school
classroom. She is also Lecturer in Social Research Methods for the
UCLA Graduate School of Education. Active in the Program Evalua-
tion division of the American Educational Research Association, she
serves as a reviewer for the journal Educational Evaluation and
Policy Planning.
1 0
Introduction
EvaluationA Management Tool
New Demands
on the Schools
The demand for
We educators
are living in a time of decreasing resources and increas-
increased
ing demands both from the communities we serve and from the wider
body politic..Our morning newspaper regularly reproaches us about
productivity.
the declining quality of American education, the unfavorable stand-
ing of American students on international assessments in mathemat-
ics and science, and the poor productivity of our schools. Each baleful
article includes test scores and statistics to bol,
its incontrovert-
ible assertions. State legislatures, in an attemp', to finance increasing
public needs with shrinking revenues, e.f.A.-:,t us to become more
productive and accountable and legislatc,
441-ions suf...h as "choice,"
"restructuring," or "school-based manag: ni,nt" as ways to leverage
the educational dollar. It is no wonder that those of us still optimistic
enough to seek out the challenges of educational leadership ask
oi'rselves with each new school improvement initiative, "Don't I
already have enough to do?"
The answer: Yes! And the solution for those of us implementing
new accountability, restructuring, and improvement mandates who
wioh to remain sane is to do with our time what legislators would have
us do with their moniesget more "bang for the buck." In industry,
increasing productivity generally means producing a larger quantity
of goods and services using the same amount of resources. In educa-
tion, productivity is not so easily defined. Increasing our productivity
1