Policy Implications of Research in Education 8 Leonidas Kyriakides Bert Creemers Evi Charalambous Equity and Quality Dimensions in Educational Eff ectiveness Policy Implications of Research in Education Volume 8 Series Editors Professor Stephen L. Jacobson, University at Buffalo – State University of New York, USA Paul W. Miller, University of Huddersfield, UK Editorial Board Prof. Helen Gunter, University of Manchester, England Prof. Stephan Huber, University of Teacher Education Central Switzerland Prof. Jonathan Jansen, University of the Free State, South Africa Prof. Karen Seashore Louis, University of Minnesota, USA Dr. Guri Skedsmo, University of Oslo, Norway Prof. Allan Walker, Hong Kong Institute of Education Scope of the Series In education, as in other fields, there are often significant gaps between research knowledge and current policy and practice. While there are many reasons for this gap, one that stands out is that policy-makers and practitioners may simply not know about important research findings because these findings are not published in forums aimed at them. Policy Implications of Research in Education aims to clearly and comprehensively present the implications for education policy and practice drawn from important lines of current education research in a manner that is accessible and useful for policy-makers, educational authorities and practitioners. More information about this series at http://www.springer.com/series/11212 Leonidas Kyriakides • Bert Creemers Evi Charalambous Equity and Quality Dimensions in Educational Effectiveness Leonidas Kyriakides Bert Creemers Department of Education Faculty of Behavioural & Social Sciences University of Cyprus University of Groningen Nicosia, Cyprus Groningen, The Netherlands Evi Charalambous Department of Education University of Cyprus Nicosia, Cyprus Policy Implications of Research in Education ISBN 978-3-319-72064-7 ISBN 978-3-319-72066-1 (eBook) https://doi.org/10.1007/978-3-319-72066-1 Library of Congress Control Number: 2017964258 © Springer International Publishing AG 2018 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Printed on acid-free paper This Springer imprint is published by Springer Nature The registered company is Springer International Publishing AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland Preface Fifteen years ago, two of us started working on the development and testing of a model of educational effectiveness in which we made use of existing theories and research on effectiveness. In the early days, educational effectiveness research (EER) contributed to the debate on whether teachers and schools can contribute to educational outcomes or whether education is unable to have any effect on promot- ing equity. Early studies show that teachers and schools can have an effect on stu- dent learning outcomes. These studies were also concerned with identifying ways to help schools in disadvantaged areas to achieve learning outcomes. Gradually, this strand of research moved on to the use of more sophisticated quantitative approaches that took into account student background characteristics and sought to determine the impact of schools on student progress rather than on the final learning outcomes. One could therefore argue that EER gradually progressed to placing more emphasis on the quality dimension of effectiveness, arguing that by promoting quality, equity may also be achieved. We did more or less the same when we attempted to develop the dynamic theory on educational effectiveness and test its validity by conducting national and international studies. To promote further development in educational effectiveness, we concentrated on the methodology and statistical techniques which were available to test the theories in practice. However, it was even more important to address the theory and evidence for improving educational effectiveness, which resulted in the development and testing in educational practice of the dynamic approach to school improvement (DASI). By examining several national and inter- national studies undertaken during the last eight years, we were able to collect evi- dence for the merits of DASI, which we used in our work on improving school effectiveness and promoting teacher professional development. Having used the dynamic approach to promote quality in education, it was time to go back to the other main question of EER and investigate whether and how edu- cation can reduce the variations in student learning outcomes, especially those attributed to the students’ background characteristics (equity dimension of educa- tional effectiveness). In this book, we provide a description of our journey and our main findings so far. Specifically, we begin with examining how the two dimensions v vi Preface of educational effectiveness, quality and equity, are related to each other, and we discuss the importance of promoting both dimensions for the benefit of each indi- vidual student and society in general (see Chap. 1). We then concentrate on identify- ing the impact of student background characteristics on their learning outcomes (see Chap. 2) and especially on how these characteristics have been specified and treated in the history of EER (see Chap. 3). Next, in Chap. 4, we present the different meth- odological approaches to measuring equity in education, and we discuss their use and merits in the different educational effectiveness studies. We eventually argue for the use of a random slope multilevel model to measure equity at classroom, school, and system level. Suggestions for conducting longitudinal studies to identify factors that explain changes in the effectiveness status of schools in terms of the quality and/or equity dimension are also provided. Following the approach described in Chap. 4, we present the results of secondary analyses of national and international studies, which aimed to investigate the relationship between quality and equity in education (see Chap. 5). In our attempt to encourage the further development and use of this methodology, we demonstrate how experimental studies can be con- ducted to find out whether and under which conditions our proposed methodology can help schools improve their effectiveness status in terms of both the quality and the equity dimensions. Specifically, in Chap. 6, we explicitly refer to the rationale and the major steps in the dynamic approach to school improvement, which was used when conducting a European experimental study aiming to promote quality and equity in schools in socially disadvantaged areas. Subsequently, in Chap. 7, we describe this experimental study and present its main results, which reveal the impact that DASI can have on promoting student learning outcomes and reducing the impact of SES on student achievement in mathematics. Finally, in Chap. 8, we discuss the implications of what we have done so far for theory, research, policy, and practice. The two dimensions of effectiveness (quality and equity) are both important, and the positive message is that they are not in conflict with each other. Educational systems, schools, and teachers can work toward achieving both dimen- sions of effectiveness for the benefit of their students and society in general. Thus, the book draws the attention of researchers, policy-makers, and practitioners to the contribution that each school can make to promoting not only quality but also equity and demonstrates that both quality and equity can be achieved by establishing more sophisticated ways of measuring a school’s contribution to promoting each dimen- sion and using an evidence-based and theory-driven approach to school improvement. In the writing of this book, we received support from many colleagues, profes- sionals in schools, and our families. We would like to make a special mention of some of them. The research assistants in our research group, and especially our postdoctoral researchers Anastasia Panayiotou and Ioulia Televantou and our PhD students Andria Dimosthenous and Elena Kokkinou, provided us with comments from the perspective of young researchers in the field of educational effectiveness and improvement. Moreover, Sheila M. Hakin not only supported us in the process of linguistic editing, but she was also critical in helping us to clarify the meaning of the book. Finally, our universities were supportive in facilitating our academic Preface vii efforts to write the book. We thank them all for their help, and we hope that they will be pleased with the final product. Of course, any mistakes that remain are ours. As mentioned previously, this book is a report of a long journey on equity in education and its relationship with quality in terms of promoting student learning outcomes. We were especially interested in discovering appropriate methods for measuring the contribution of schools in promoting equity in education and whether the dynamic approach to school improvement could contribute to promoting both quality and equity in education. We have found some positive answers, but still there are many questions left. We welcome comments, criticism, and contributions to the further development of research on educational effectiveness and school improvement from readers with different perspectives in education. We hope that you can join us in our journey to make education effective and more equitable. Nicosia, Cyprus Leonidas Kyriakides Evi Charalambous Groningen, The Netherlands Bert Creemers Contents 1 Quality and Equity Dimensions of Educational Effectiveness: An Introduction ......................................................................................... 1 1.1 Introduction ........................................................................................ 1 1.2 M easuring School Effectiveness in Terms of Quality and Equity: The Contribution of Educational Effectiveness Research ...................................................................... 5 1.2.1 Student Outcomes: The Cognitive Domain of Learning ............................................................................. 6 1.2.2 Student Outcomes: Beyond the Cognitive Domain of Learning ............................................................................. 7 1.2.3 Dimensions of Effectiveness: Quality and Equity ................. 8 1.3 E stablishing Evaluation Mechanisms to Measure School Effectiveness in Terms of Quality and Equity ................................... 10 1.3.1 Implicit Operational Definitions for Measuring the Quality Dimension of School Effectiveness .................... 11 1.3.2 Establishing Evaluation Mechanisms Measuring the Equity Dimension of Effectiveness .................................. 13 1.3.3 Use of Teacher and School Evaluation Mechanisms for Formative Purposes .......................................................... 14 1.4 Aims and Outline of the Book ........................................................... 15 References ................................................................................................... 17 2 The Impact of Student Characteristics on Student Achievement: A Review of the Literature ............................................... 23 2.1 Introduction ........................................................................................ 23 2.2 T he Effect of Socioeconomic Background Factors on Student Achievement ...................................................................................... 24 2.3 S tudent-Level Factors Beyond SES Associated with Student Learning Outcomes Included in the Dynamic Model of Educational Effectiveness .............................................................. 30 ix x Contents 2.3.1 Sociocultural and Economic Background Variables Emerging from the Sociological Perspective of EER ............ 30 2.3.2 Background Variables That Emerged from the Psychological Perspective of EER .......................... 34 2.3.3 Variables Related to Specific Learning Tasks Emerging from the Psychological Perspective of EER .......................... 39 2.4 Conclusions ........................................................................................ 41 References ................................................................................................... 42 3 Investigating the Quality and Equity Dimensions: A Critical Review of Literature on Educational Effectiveness ............................... 51 3.1 Introduction ........................................................................................ 51 3.2 H istory of Educational Effectiveness Research and Its Attention to Quality ................................................................ 52 3.2.1 First Phase: Establishing the Field by Showing That School Matters ............................................................... 53 3.2.2 Second Phase: Searching for Factors Associated with Student Outcomes .......................................................... 54 3.2.3 Third Phase: Development of Theoretical Models ................ 55 3.2.4 Fourth Phase: Analysing in More Detail the Complex Nature of Educational Effectiveness ...................................... 58 3.3 Methodological Developments Promoting Theory and Research on the Quality Dimension of Effectiveness ................. 59 3.4 Conclusions ........................................................................................ 62 References ................................................................................................... 64 4 Methodological Approaches to Measuring Equity in Education .......... 69 4.1 Introduction ........................................................................................ 69 4.2 A Gini-Type Index for Measuring Equity in Schools ........................ 70 4.3 I nvestigating the Impact of Student Background Factors on Student Achievement .................................................................... 73 4.4 E xamining the Reduction of the Impact of Student Background Factors on Student Achievement ................................... 76 4.5 Determining Differential Teacher/School Effectiveness in Relation to Student Background Characteristics ........................... 77 4.5.1 Strengths and Limitations of the Approach Used to Measure Differential School and Teacher Effectiveness ... 79 4.6 M easuring Changes in the Effectiveness Status of Schools and Teachers in Promoting Quality and Equity: A Further Extension of the Approach Used to Identify Differential Effects ...... 80 4.6.1 Searching for Stability and Changes in the Effectiveness Status of Schools: Methods of the Study ............................... 81 4.6.2 Investigating for Stability and Changes in the Effectiveness Status of Schools: Main Results of the Study ............................................................................ 84