Methodological Advances in Educational Effectiveness Research (Quantitative Methodology Series)

377 Pages·2010·2.15 MB·English

by  Bert P.M. Creemers

Methodological Advances
in Educational
Effectiveness Research
Bert P.M. Creemers,
Leonidas Kyriakides and
Pam Sammons
First published 2010
by Routledge
2 Park Square, Milton Park, Abingdon, Oxon OX14 4RN
Simultaneously published in the USA and Canada
by Routledge
270 Madison Avenue, New York, NY 10016
Routledge is an imprint of the Taylor & Francis Group, an informa business
This edition published in the Taylor & Francis e-Library, 2010.
To purchase your own copy of this or any of Taylor & Francis or Routledge’s
collection of thousands of eBooks please go to www.eBookstore.tandf.co.uk.
© 2010 Bert P.M. Creemers, Leonidas Kyriakides and Pam Sammons
for selection and editorial material. Individual contributions, the
contributors.
All rights reserved. No part of this book may be reprinted or
reproduced or utilized in any form or by any electronic, mechanical, or
other means, now known or hereafter invented, including photocopying
and recording, or in any information storage or retrieval system,
without permission in writing from the publishers.
British Library Cataloguing in Publication Data
A catalogue record for this book is available from the British Library
Library of Congress Cataloging-in-Publication Data
Creemers, Bert P.M.
Methodological advances in educational effectiveness research/Bert
P.M. Creemers, Leonidas Kyriakides, and Pam Sammons. 1st ed.
p. cm.
Includes bibliographical references and index.
1. Education Research Methodology. I. Kyriakides, Leonidas.
II. Sammons, Pam. III. Title.
LB1028.C736 2010
370.7′2 dc22 2009048160
ISBN 0-203-85100-5 Master e-book ISBN
ISBN10: 0 415 48175 9 (hbk)
ISBN10: 0 415 48176 7 (pbk)
ISBN10: 0 203 85100 5 (ebk)
ISBN13: 978 0 415 48175 5 (hbk)
ISBN13: 978 0 415 48176 2 (pbk)
ISBN13: 978 0 203 85100 5 (ebk)

Contents
List of contributors vii
List of figures viii
List of tables x
Preface xii
PART A
The state of the art of Educational Effectiveness
Research: challenges for research methodology
1 Background to Educational Effectiveness Research 3
2 Methodological issues in Educational Effectiveness Research 19
3 Theory and research: the problem of causality 37
4 Theory-driven evaluation studies: establishing links between
research, policy and practice 59
PART B
The contribution of different methodological
orientations: the development of educational
effectiveness research
5 Longitudinal designs 77
JAN-ERIC GUSTAFSSON
6 Experimental studies in education 102
ROBERT E. SLAVIN
7 Applications of mixed methods to the field of Educational
Effectiveness Research 115
CHARLES TEDDLIE AND PAM SAMMONS
vi Contents
8 Using Item Response Theory to measure outcomes and
factors: an overview of Item Response Theory models 153
NORMAN VERHELST
9 IRT models: parameter estimation, statistical testing and
application in EER 183
NORMAN VERHELST
10 Using Generalizability Theory 219
GEORGE A. MARCOULIDES AND LEONIDAS KYRIAKIDES
11 Multilevel modelling 246
HANS LUYTEN AND PAM SAMMONS
12 Structural Equation Modelling techniques 277
GEORGE A. MARCOULIDES AND LEONIDAS KYRIAKIDES
13 Meta-analyses of effectiveness studies 303
LEONIDAS KYRIAKIDES AND BERT P. M. CREEMERS
PART C
Methodological advances and EER: retrospect and
prospect
14 Conclusions for the development of EER 327
Appendices to Chapter 10 348
Appendices to Chapter 12 351
Index 353
Contributors
Authors of the book
Bert P.M. Creemers, Centre for Learning and Teaching, University of
Groningen. [email protected]
Leonidas Kyriakides, Department of Education, University of Cyprus.
[email protected]
Pam Sammons, Department of Education, University of Oxford.
[email protected]
Authors of/contributors to Part B of the book
Jan-Eric Gustafsson, University of Gothenburg, Sweden.
[email protected]
Hans Luyten, University of Twente, Department of Educational
Organisation and Management. [email protected]
George A. Marcoulides, Graduate School of Education, University of
California, Riverside. [email protected]
Robert E. Slavin, Johns Hopkins University and University of York.
[email protected]
Charles Teddlie, Department of Educational Theory, Policy and Practice,
Louisiana State University. [email protected]
Norman Verhelst, CITO, the Netherlands. [email protected]
Figures
4.1 A framework for conducting theory-driven evaluation studies
in educational effectiveness 62
5.1 An autoregressive model for repeated measures 95
5.2 An autoregressive and cross-lagged model for bivariate
measures 95
5.3 An autoregressive, cross-lagged model for development of
reading and cognitive abilities 97
7.1 The inductive–deductive research cycle (cycle of scientific
methodology) 117
7.2 Graphic illustration of sequential mixed designs 120
7.3 Summary of VITAE sampling design 138
7.4 Teachers’ selves as factors influencing effectiveness 141
8.1 Graphical representation of scalogram analysis 158
8.2 Item response functions in the Rasch model 159
8.3 Two items with equal difficulty but different discrimination 165
8.4 Two items in the 3PLM with equal discrimination and
difficulty 167
8.5 The normal ogive model using latent responses 168
8.6 Similarity of the normal ogive and the logistic models 170
8.7 Category response functions in the NRM 173
8.8 Two examples of the partial credit model 175
8.9 Category response functions in the GRM 178
9.1 Simple two-group designs 184
9.2 Anchor test design 185
9.3 Two block-interlaced anchoring designs 185
9.4 Test design for developmental studies 186
9.5 A design suited to longitudinal studies 187
9.6 DIF of a PISA mathematics item (DIF is with respect
to cycle) 202
9.7 Factor pattern of reading and listening items 207
9.8 Conditional bias of the ML and the WML estimators in the
Rasch model 210
Figures ix
10.1 A Structural Equation Model for estimating the relative
generalizability coefficient in a two-facet crossed design 234
12.1 Commonly used symbols to represent SEM models in path
diagrams 279
12.2 Confirmatory Factor Analysis model with two latent variables 280
12.3 Example structural regression model 283
12.4 Example longitudinal achievement model 297
13.1 The dynamic model of educational effectiveness 315
14.1 A conceptual road map for conducting research 333