Synergies for better learning : an international perspective on evaluation and assessment

721 Pages·2013·10.607 MB·English

by  OECD

OECD Reviews of Evaluation and Assessment in Education O
E
C
Synergies for Better Learning D
R
e OECD Reviews of Evaluation
v
ie
AN INTERNATIONAL PERSPECTIVE ON EVALUATION AND ASSESSMENT w
and Assessment in Education
s
o
How can assessment and evaluation policies work together more effectively to improve student outcomes in f E
v
a
primary and secondary schools? Countries increasingly use a range of techniques for student assessment, lu Synergies for Better Learning
a
teacher appraisal, school evaluation, school leader appraisal and education system evaluation. However, they t
io
often face diffi culties in implementing evaluation and assessment policies. This may arise as a result of poor n
a
policy design, lack of analysis of unintended consequences, little capacity for school agents to put evaluation n
procedures into practice, lack of an evaluation culture, or defi cient use of evaluation results. d A AN INTERNATIONAL PERSPECTIVE ON EVALUATION
s
This report provides an international comparative analysis and policy advice to countries on how evaluation se AND ASSESSMENT
s
and assessment arrangements can be embedded within a consistent framework to improve the quality, equity s
m
and effi ciency of school education. It builds upon a major 3-year review of evaluation and assessment policies e
n
in 28 countries, the OECD Review on Evaluation and Assessment Frameworks for Improving School Outcomes. t in
As well as analysing strengths and weaknesses of different approaches, the report provides recommendations E
for improvement including how results should be incorporated into policy and practice. du
c
a
Contents tio
n
Chapter 1. The focus on evaluation and assessment
Chapter 2. Trends in evaluation and assessment
Chapter 3. The evaluation and assessment framework: Embracing a holistic approach
Chapter 4. Student assessment: Putting the learner at the centre
Chapter 5. Teacher appraisal: Enhancing teacher professionalism
Chapter 6. School evaluation: From compliancy to quality
Chapter 7. The appraisal of school leaders: Fostering pedagogical leadership in schools
Chapter 8. Education system evaluation: Informing policies for system improvement
www.oecd.org/edu/evaluationpolicy
S
y
n
e
r
g
ie
s
fo
r
B
e
t
t
e
r
L
e
a
r
Consult this publication on line at http://dx.doi.org/10.1787/9789264190658-en. n
in
g
This work is published on the OECD iLibrary, which gathers all OECD books, periodicals and statistical databases.
Visit www.oecd-ilibrary.org for more information.
ISBN 978-92-64-19064-1 -:HSTCQE=V^U[YV:
91 2013 02 1 P
912013021fin.indd 1 22-Mar-2013 10:01:47 AM

OECD Reviews of Evaluation and Assessment in Education
Synergies
for Better Learning
AN INTERNATIONAL PERSPECTIVE
ON EVALUATION AND ASSESSMENT
This work is published on the responsibility of the Secretary-General of the OECD. The
opinions expressed and arguments employed herein do not necessarily reflect the official
views of the Organisation or of the governments of its member countries.
This document and any map included herein are without prejudice to the status of or
sovereignty over any territory, to the delimitation of international frontiers and boundaries
and to the name of any territory, city or area.
ISBN 978-92-64-19064-1 (print)
ISBN 978-92-64-19065-8 (PDF)
http://dx.doi.org/10.1787/9789264190658-en
Series: OECD Reviews of Evaluation and Assessment in Education
ISSN 2223-0947 (print)
ISSN 2223-0955 (online)
The statistical data for Israel are supplied by and under the responsibility of the relevant Israeli authorities. The use
of such data by the OECD is without prejudice to the status of the Golan Heights, East Jerusalem and Israeli
settlements in the West Bank under the terms of international law.
Photo credits: Cover © iQoncept - Fotolia.com, © AKS - Fotolia.com, © Sergej Khackimullin - Fotolia.com.
Corrigenda to OECD publications may be found on line at: www.oecd.org/publishing/corrigenda.
© OECD 2013
You can copy, download or print OECD content for your own use, and you can include excerpts from OECD publications, databases and
multimedia products in your own documents, presentations, blogs, websites and teaching materials, provided that suitable
acknowledgment of the source and copyright owner is given. All requests for public or commercial use and translation rights should be
submitted to [email protected]. Requests for permission to photocopy portions of this material for public or commercial use shall be
addressed directly to the Copyright Clearance Center (CCC) at [email protected] or the Centre français d'exploitation du droit de copie
(CFC) at [email protected].
3
FOREWORD –

Foreword
Authentic, valid and reliable evaluation and assessment, those which lead to the
improvement of educational practices at all levels and lift student learning, are central to
establishing a high-performing education system. They are also instrumental in
recognising and rewarding the work of educational practitioners and in certifying the
learning of students. Promoting evaluation and assessment is clearly in the interest of
students and their families, educational practitioners and education systems. As a result,
more and more countries embark on ambitious school reform programmes which include
a strong element of evaluation and assessment. This can consolidate the evaluation
culture in education systems and reinforces the role of evaluation and assessment
frameworks in driving the reform agenda.
In late 2009, the OECD Education Policy Committee embarked on a comprehensive
international review of evaluation and assessment policies, the OECD Review on
Evaluation and Assessment Frameworks for Improving School Outcomes. Its goal was to
provide analysis and policy advice to countries on how evaluation and assessment
arrangements can be embedded within a consistent framework that can bring about real
gains in performance across the school system. In addition to this publication, the Review
generated 25 reports by participating countries, 15 reports by external review teams
(released as a publication series, OECD Reviews of Evaluation and Assessment in
Education) and several research papers (all available on the OECD website at
www.oecd.org/edu/evaluationpolicy). This OECD Review provides an unparalleled
comprehensive analysis of evaluation and assessment policy issues at international level.
OECD work helps countries to learn from one another. It can also highlight issues
and explore policy options that may be difficult to raise in national debates. Both of
these elements clearly underpin this report and the work behind it. The direct
involvement of countries has been crucial to the process. The 28 countries that were
actively engaged committed substantial resources and opened their evaluation and
assessment policies to external review and debate. This collaborative approach enabled
countries to learn more about themselves and to add to the broader knowledge base by
sharing evidence on the impact of policy reforms and the circumstances under which
they work best.
The Review was overseen by the OECD Group of National Experts (GNE) on
Evaluation and Assessment, which was established as a subsidiary body of the Education
Policy Committee. The GNE on Evaluation and Assessment was chaired by Mr. Gábor
Halász, Professor, Eötvös Loránd University, Hungary, and had as Vice-Chairs
Mr. Éamonn Murtagh, Assistant Chief Inspector, Department of Education and Skills,
Ireland, and Mr. Kwangho Kim, Research Fellow, Korean Educational Development
Institute, Korea. We are grateful for their insightful and committed guidance.

SYNERGIES FOR BETTER LEARNING: AN INTERNATIONAL PERSPECTIVE ON EVALUATION AND ASSESSMENT © OECD 2013
4
– FOREWORD

The Review benefited substantially from the involvement of organisations
representing students, parents, teachers, school leaders, schools, teacher educators,
researchers and employers. Their representatives served on national committees, prepared
written submissions, met with review teams and participated in conferences and meetings
of the GNE on Evaluation and Assessment. The Review also benefited from the
contribution of the Business and Industry Advisory Committee to the OECD (BIAC) and
the Trade Union Advisory Committee to the OECD (TUAC), as well as other
international organisations interested in evaluation and assessment policy, including the
European Commission, Eurydice, the Standing International Conference of Inspectorates
(SICI), UNESCO and the World Bank.
Annex A details the many people and organisations who contributed to the Review as
national co-ordinators, members of the GNE on Evaluation and Assessment, members of
country review teams, and authors of country background reports and commissioned
research papers – more than 120 people in all. We are fortunate to have benefited from
their expertise and are appreciative of their commitment to the Review. The 15 review
visits involved 30 external reviewers with a range of research and policy backgrounds.
Overall, the external review teams visited about 90 schools and met with about 2 800
individuals on which the country review findings were based. We thank them all for their
valuable contributions to the collective knowledge base.
Within the Directorate for Education and Skills, the Review was carried out by the
Education and Training Policy Division under the leadership of Deborah Roseveare (from
October 2009 until January 2012) and by the Early Childhood and Schools Division
under the leadership of Michael Davidson (from February 2012 on). Within the OECD
Secretariat, Paulo Santiago (co-ordinator), Deborah Nusche and Claire Shewbridge were
responsible for the Review and the preparation of this report. They also took
responsibility for individual country reviews. Heike-Daniela Herzog took responsibility
for the administrative work within the Review, the organisation of meetings and
communication with the countries. A number of other colleagues contributed to both the
Review and this report (see Acknowledgements below). A larger group of colleagues
within the OECD provided advice at key stages. In particular, collaboration was
established with the Programme for International Student Assessment (PISA), the
Teaching and Learning International Survey (TALIS), CERI’s Innovative Learning
Environments (ILE) project, the INES Network for the Collection and the Adjudication of
System-Level Descriptive Information on Educational Structures, Policies and Practices
(NESLI), CERI’s work on Longitudinal Information Systems as part of its Innovation
Strategy for Education and Training, CERI’s work on Governing Complex Education
Systems and the Centre for Effective Learning Environments’ (CELE) work on
evaluating quality in educational facilities.
This report was released in Oslo on 11 April 2013 at an International Conference
jointly sponsored by the Ministry of Education and Research of Norway, the Norwegian
Directorate for Education and Training, and the OECD.
The OECD intends to maintain the momentum of its work on evaluation and
assessment policies and to build on the Review on Evaluation and Assessment
Frameworks for Improving School Outcomes and this report.

SYNERGIES FOR BETTER LEARNING: AN INTERNATIONAL PERSPECTIVE ON EVALUATION AND ASSESSMENT © OECD 2013
5
ACKNOWLEDGEMENTS –

ACKNOWLEDGEMENTS
The authors of this report are Deborah Nusche, Thomas Radinger, Paulo Santiago
(co-ordinator) and Claire Shewbridge of the Directorate for Education and Skills, with the
following distribution of responsibilities as lead authors:
Chapter 1. The focus on evaluation and assessment Paulo Santiago
Chapter 2. Trends in evaluation and assessment Paulo Santiago
Chapter 3. The evaluation and assessment framework Paulo Santiago
Chapter 4. Student assessment Deborah Nusche
Chapter 5. Teacher appraisal Deborah Nusche
Chapter 6. School evaluation Claire Shewbridge
Chapter 7. The appraisal of school leaders Thomas Radinger
Chapter 8. Education system evaluation Claire Shewbridge
Heike-Daniela Herzog was responsible for the production and layout of the report and
contributed to its editing and proofreading. Valuable comments on draft chapters were
provided by members of the OECD Secretariat, members of the Group of National
Experts on Evaluation and Assessment, members of the OECD Education Policy
Committee, national co-ordinators of participating countries, and researchers and
international agencies associated with the work. The team of authors is grateful to
individual experts who contributed to the country-specific OECD Reviews of Evaluation
and Assessment in Education, whose expertise and analysis have fed into this report (they
are listed in Table A.3 of Annex A).
Thanks are due to the many people who worked on this Review at different stages of
its development, in addition to its core members (Heike-Daniela Herzog, Deborah
Nusche, Paulo Santiago, Claire Shewbridge): Morten Rosenkvist (February to July 2010)
on secondment from the Ministry of Education and Research of Norway; Stefanie Dufaux
(September 2010 to August 2011) as a Fellow of the Carlo Schmid Programme and as a
Consultant; and Thomas Radinger (September 2011 to September 2012) as a Fellow of the
Carlo Schmid Programme and as a Consultant. They provided substantial input into the
Review’s knowledge base through their analytical work. Thomas Radinger also assumed
responsibility for the analysis on school leader appraisal and co-ordinated the preparation
of the tables summarising features of evaluation and assessment frameworks. Both
Stefanie Dufaux and Thomas Radinger took responsibility for statistical work within the
Review. Violaine Faubert, Marlène Isoré, Allison Morris and Anne Nayral de Puybusque
provided research assistance summarising key areas of the literature on evaluation and
assessment during their internships at the OECD. Janet Looney, as an external Consultant,
prepared research papers as background analytical work for the Review. Francisco
Benavides and Diana Toledo, of the OECD Secretariat, contributed as experts to the
OECD Reviews of Evaluation and Assessment in Chile and Mexico, respectively.
SYNERGIES FOR BETTER LEARNING: AN INTERNATIONAL PERSPECTIVE ON EVALUATION AND ASSESSMENT © OECD 2013
6
– ACKNOWLEDGEMENTS

Sabrina Leonarduzzi (from June until December 2009) and Sara-Jayne Moss (from
January until June 2010) provided administrative support for the Review. Cassandra
Davis (Directorate for Education and Skills) and Anne-Lise Prigent (Public Affairs and
Communications Directorate) provided advice on the production of the report and
dissemination activities. Brian Keeley (Public Affairs and Communications Directorate)
prepared the executive summary of this report. Therese Walsh and Damian Garnys (both
from the Public Affairs and Communications Directorate) provided editorial assistance.
We are also grateful to the German Academic Exchange Service (DAAD) and the
German National Academic Foundation for sponsoring Stefanie Dufaux’s (September
2010 – March 2011) and Thomas Radinger’s (September 2011 – June 2012)
appointments with the OECD Secretariat as Fellows of the Carlo Schmid Programme.
Gratitude is also extended to the Ministry of Education and Research of Norway for
supporting the secondment of Morten Rosenkvist with the OECD Secretariat and for
hosting the International Conference to launch this report (11-12 April 2013).

This book has...
StatLinks2
A service that delivers Excel® files
from the printed page!
Look for the StatLinks at the bottom right-hand corner of the tables or graphs in this book.
To download the matching Excel® spreadsheet, just type the link into your Internet browser,
starting with the http://dx.doi.org prefix.
If you’re reading the PDF e-book edition, and your PC is connected to the Internet, simply
click on the link. You’ll find StatLinks appearing in more OECD books.

SYNERGIES FOR BETTER LEARNING: AN INTERNATIONAL PERSPECTIVE ON EVALUATION AND ASSESSMENT © OECD 2013
7
TABLE OF CONTENTS –
Table of contents
Executive summary ................................................................................................................................... 13
Overview .................................................................................................................................................... 17
Chapter 1. The focus on evaluation and assessment ............................................................................... 29
The growing importance of evaluation and assessment .......................................................................... 30
Methodology ........................................................................................................................................... 30
Organisation of the report ....................................................................................................................... 32
Chapter 2. Trends in evaluation and assessment .................................................................................... 35
Introduction ............................................................................................................................................. 36
Trends in evaluation and assessment ....................................................................................................... 36
Contextual developments shaping evaluation and assessment ................................................................ 45
References ............................................................................................................................................... 53
Chapter 3. The evaluation and assessment framework: Embracing a holistic approach ................... 57
Introduction ............................................................................................................................................. 58
Definitions and scope .............................................................................................................................. 59
Educational context ................................................................................................................................. 61
Governance.............................................................................................................................................. 62
Overarching reference: Goals for student learning ................................................................................. 73
Design ..................................................................................................................................................... 78
Capacity for evaluation and assessment .................................................................................................. 97
Use of results ......................................................................................................................................... 102
Implementation of evaluation and assessment policies ......................................................................... 108
Pointers for future policy development ................................................................................................. 117
References ............................................................................................................................................. 129
Chapter 4. Student assessment: Putting the learner at the centre ...................................................... 139
Introduction ........................................................................................................................................... 140
Analytical approach ............................................................................................................................... 140
Impact, drivers and contextual developments ....................................................................................... 144
Governance............................................................................................................................................ 152
Procedures ............................................................................................................................................. 170
Capacity ................................................................................................................................................. 191
Reporting and use of results .................................................................................................................. 202
Pointers for future policy development ................................................................................................. 213
References ............................................................................................................................................. 225
Annex 4.A1 Features of student assessment frameworks in lower secondary education ..................... 239
Annex 4.A2 Features of student assessment frameworks in primary and upper secondary education .. 269
SYNERGIES FOR BETTER LEARNING: AN INTERNATIONAL PERSPECTIVE ON EVALUATION AND ASSESSMENT © OECD 2013
8
– TABLE OF CONTENTS
Chapter 5. Teacher appraisal: Enhancing teacher professionalism ................................................... 271
Introduction ........................................................................................................................................... 272
Analytical approach ............................................................................................................................... 272
Impact, drivers and contextual developments ....................................................................................... 274
Governance............................................................................................................................................ 280
Procedures ............................................................................................................................................. 292
Capacity ................................................................................................................................................. 312
Use of results ......................................................................................................................................... 322
Pointers for future policy development ................................................................................................. 332
References ............................................................................................................................................. 343
Annex 5.A Features of teacher appraisal frameworks........................................................................... 353
Chapter 6. School evaluation: From compliancy to quality ................................................................ 383
Introduction ........................................................................................................................................... 384
Analytical approach ............................................................................................................................... 384
Impact, drivers and contextual developments ....................................................................................... 386
Governance............................................................................................................................................ 393
Procedures ............................................................................................................................................. 414
Capacity ................................................................................................................................................. 432
Reporting and use of results .................................................................................................................. 445
Pointers for future policy development ................................................................................................. 463
References ............................................................................................................................................. 473
Chapter 7. The appraisal of school leaders: Fostering pedagogical leadership in schools................ 485
Introduction ........................................................................................................................................... 486
Analytical approach ............................................................................................................................... 486
Impact, drivers and contextual developments ....................................................................................... 496
Governance............................................................................................................................................ 506
Procedures ............................................................................................................................................. 517
Capacity ................................................................................................................................................. 539
Use of results ......................................................................................................................................... 543
Pointers for future policy development ................................................................................................. 550
References ............................................................................................................................................. 561
Annex 7.A Features of school leader appraisal frameworks ................................................................. 569
Chapter 8. Education system evaluation: Informing policies for system improvement .................... 583
Introduction ........................................................................................................................................... 584
Analytical approach ............................................................................................................................... 584
Impact, drivers and contextual developments ....................................................................................... 587
Governance............................................................................................................................................ 590
Procedures ............................................................................................................................................. 595
Capacity ................................................................................................................................................. 618
Reporting and use of results .................................................................................................................. 626
Pointers for future policy development ................................................................................................. 639
References ............................................................................................................................................. 649
Annex A. How the Review on Evaluation and Assessment Frameworks for Improving School
Outcomes was conducted ............................................................................................................... 657
Annex B. Summary of policy directions from the Review on Evaluation and Assessment
Frameworks for Improving School Outcomes ............................................................................. 667
SYNERGIES FOR BETTER LEARNING: AN INTERNATIONAL PERSPECTIVE ON EVALUATION AND ASSESSMENT © OECD 2013