Evaluating the Validity of Mcas Scores as an Indicator of Teacher Effectiveness

169 Pages·2014·1.28 MB·English

by  Jenna M. Copella

University of Massachusetts Amherst
ScholarWorks@UMass Amherst
Open Access Dissertations
2-2013
Evaluating the Validity of Mcas Scores as an
Indicator of Teacher Effectiveness
Jenna M. Copella
University of Massachusetts Amherst, [email protected]
Follow this and additional works at:https://scholarworks.umass.edu/open_access_dissertations
Part of theEducation Commons
Recommended Citation
Copella, Jenna M., "Evaluating the Validity of Mcas Scores as an Indicator of Teacher Effectiveness" (2013).Open Access Dissertations.
682.
https://doi.org/10.7275/xjh7-3c87https://scholarworks.umass.edu/open_access_dissertations/682
This Open Access Dissertation is brought to you for free and open access by ScholarWorks@UMass Amherst. It has been accepted for inclusion in
Open Access Dissertations by an authorized administrator of ScholarWorks@UMass Amherst. For more information, please contact
[email protected].
EVALUATING THE VALIDITY OF MCAS SCORES AS AN INDICATOR OF TEACHER
EFFECTIVENESS.

A Dissertation Presented
By
JENNA M. COPELLA

Submitted to the Graduate School of the
University of Massachusetts Amherst in partial fulfillment
of the requirements for the degree of

DOCTOR OF EDUCATION

February 2013

School of Education
Educational Policy Research and Administration
Psychometric Methods, Educational Statistics, and Research Methods
© Copyright by Jenna M. Copella 2013
All Rights Reserved
EVALUATING THE VALIDITY OF MCAS SCORES AS AN INDICATOR OF TEACHER
EFFECTIVENESS.

A Dissertation Presented
By
JENNA M. COPELLA

Approved as to style and content by:

____________________________________
Lisa A. Keller, Chair

____________________________________
Stephen G. Sireci, Member

____________________________________
George R. Milne, Member

______________________________________
Christine B. McCormick, Dean
School of Education
DEDICATION
For my children: Riley, Aidan, and Caitlin Copella. You bring meaning, purpose,
and joy to my life. You are the three best things that have ever happened to me. I love
you.
ACKNOWLEDGEMENTS
I would like to thank my advisor and committee chair, Dr. Lisa A. Keller. This
work would not have been possible without her guidance and encouragement. Lisa, I
would like to take this opportunity to say thank you. Thank you for sharing your time and
knowledge with me, thank you for knowing when to push, thank you for understanding
all of the things only another mother could understand, thank you for not judging, thank
you for being my advisor, and thank you for being a friend. I hope you know how
grateful I am and just how deeply I appreciate you.
I would also like to thank the rest of my department at UMASS; Ron Hambleton,
Steve Sireci, Craig Wells, Jennifer Randall, April Zenisky, and Peg Louraine. They are a
wonderful group of people. Steve Sireci introduced me to psychometrics, for which I will
be forever grateful. I would never have found my way to UMASS without Steve. Ron
Hambleton was a treasure trove of knowledge, anecdotes, and advice. I will hold dear
many memories of Ron. Craig Wells, Jennifer Randall, and April Zenisky were a
pleasure to work with and learn from every day. Peg Louraine does everything humanly
possible to ease the way for the students in the program. The care and support these
people provide their students is amazing. I am so lucky to have been a part of it. My life,
both professionally and personally, has been enriched because of my time at REMP.
Thank you to Steve Sireci and George Milne for serving on my dissertation
committee. Your comments, advice, and feedback have been invaluable.
Finally, thank you to my family. I have received so much love and support from
everyone. I need to send a special thank you to my mother, Terri Sisson, who has
supported me every step of this journey. I simply do not know what I would do without
v
her. And to my Aunty Jo, Joanne Coyle, who has always been there for me. My Mom and
Aunty Jo have listened tirelessly to me talk about school and, more recently, my
dissertation. They have both spent hours proofreading. And they both helped me with the
dirtiest job of all – email addresses! Without them I would still be copying and pasting!
They are incredible women. So to Mom and Aunty Jo… thank you for everything.

vi
ABSTRACT
EVALUATING THE VALIDITY OF MCAS SCORES AS AN INDICATOR OF
TEACHER EFFECTIVENESS
FEBRUARY 2013
JENNA M. COPELLA, A.A., SPRINGFIELD TECHNICAL COMMUNITY COLLEGE
B.A., WESTFIELD STATE UNIVERSITY
Ed.D., UNIVERSITY OF MASSACHUSETTS AMHERST
Directed by: Lisa A. Keller
The Massachusetts Department of Secondary and Elementary Education (DESE)
has implemented an Educator Evaluation Framework that requires MCAS scores be used
as a significant indicator of teacher effectiveness when available. This decision has
implications for thousands of Massachusetts public school teachers. To date, DESE has
not provided evidence to support the validity of using MCAS scores to make
interpretations about teacher effectiveness. A review of the literature reveals much
variation in the degree to which teachers use state-adopted content standards to plan
instruction. The findings in the literature warrant investigation into teacher practice
among Massachusetts public school teachers. The research questions for this study will
be: 1.) Are there variations in the degree to which Massachusetts public school teachers
use the Curriculum Frameworks to plan Math instruction?; and 2.) Is MCAS as an
instrument sensitive enough to reflect variations in teacher practice in the student’s
scores? A survey of Massachusetts public school principals and Math teachers, grades
three through eight, investigated the research questions. Survey results revealed that
Massachusetts teachers use the Curriculum Frameworks to plan instruction to varying
degrees. Survey results also suggest a lack of relationship between teacher practice
vii
related to the use of the Curriculum Frameworks and student MCAS scores. These
findings suggest MCAS scores may not be an appropriate indicator of teacher
effectiveness; however, there are limitations to the study that require further investigation
into these questions.

viii
TABLE OF CONTENTS
ACKNOWLEDGEMENTS …...………………………………………………………….v
ABSTRACT ……………...……………………………………………………………..vii
LIST OF TABLES ………………..………………………………………………….......xi
LIST OF FIGURES ..........................................................................................................xii
CHAPTER
1. INTRODUCTION ………………………………………………………………..……1
2. LITERATURE REVIEW ………………………………………………………….…17
The Standards ……………………………………………………………………20
Instructional Alignment …………………………………………………………27
Proposed Alignment Frameworks ……………………………………………….37
Teacher Practice …………………………………………………………………43
Research Questions ……………………………………………………………...47

3. METHODS …………………………………………………………………………...52
Pilot Study ………………………………………………………………………52
Method …………………………………………………………………..53
Full Study ………………………………………………………………………..54
Survey Construction ……………………………………………………..54
Sample …………………………………………………………………...56
Surveys …………………………………………………………………..57
Analyses …………………………………………………………………59
Descriptive Statistics …………………………………………….60
Pearson Product Moment Correlation …………………………...60
Analysis of Variance …………………………………………….61
Linear Regression ……………………………………………….61

4. RESULTS …………………………………………………………………………….65
Pilot Study ……………………………………………………………………….65
Full Study ………………………………………………………………………..69

5. DISCUSSION ……………………………………………………………………….100
APPENDICES
A. PILOT STUDY RESPONSES …………………………………………………123
ix