Schools, Classrooms, and Pupils. International Studies of Schooling from a Multilevel Perspective

259 Pages·1991·16.299 MB·English

by  Stephen W. Raudenbush, J. Douglas Willms

Schools, Classrooms,
and Pupils
International Studies of Schooling
from a Multilevel Perspective
Edited by
Stephen W. Raudenbush
College of Education
Michigan State University
East Lansing, Michigan
J. Douglas Willms
Centre for Policy Studies in Education
University of British Columbia
Vancouver, British Columbia, Canada
and Centre for Educational Sociology
University of Edinburgh
Edinburgh, Scotland
<3
ACADEMIC PRESS, INC.
Harcourt Brace Jovanovich, Publishers
San Diego New York Boston London Sydney Tokyo Toronto
This book is printed on acid-free paper. ©
Copyright © 1991 by ACADEMIC PRESS, INC.
All Rights Reserved.
No part of this publication may be reproduced or transmitted in
any form or by any means, electronic or mechanical, including
photocopy, recording, or any information storage and retrieval
system, without permission in writing from the publisher.
Academic Press, Inc.
San Diego, California 92101
United Kingdom Edition published by
Academic Press Limited
24-28 Oval Road, London NW1 7DX
Library of Congress Cataloging-in-Publication Data
Schools, pupils, and classrooms : international studies of schooling
from a multilevel perspective / [edited by] Stephen W. Raudenbush,
J. Douglas Willms.
p. cm.
Includes index.
ISBN 0-12-582910-8
1. School supervision-Congresses. 2. Academic achievement-
-Congresses. 3. Education-Research-Congresses. I. Raudenbush,
Stephen W. II. Willms, Jon Douglas.
LB22806.4.S35 1991
371.2-dc20 90-25507
CIP
PRINTED IN THE UNITED STATES OF AMERICA
90 91 92 93 9 8 7 6 5 4 3 2 1
Contributors
Numbers in parentheses indicate the pages on which the authors' contributions begin.
Murray Aitkin (25), Department of Statistics, Tel Aviv University, Ramat
Aviv 69 978, Israel
R. J. Bosker (101), Department of Education, University of Twente, 7500
AE Enschede, The Netherlands
Anthony S. Bryk (185), Department of Education, University of Chicago,
Chicago, Illinois 60637
Carol T. Fitz-Gibbon (67), School of Education, University of Newcastle
Upon Tyne, Newcastle Upon Tyne, NE1 7RU, Great Britain
Ken Frank (185), Department of Education, University of Chicago, Chi-
cago, Illinois 60637
Adam Gamoran (37), Wisconsin Center for Education Research, Univer-
sity of Wisconsin-Madison, Madison, Wisconsin 53706
H. Guldemond (101), Institut voor Onderwijsonderzoek, Rijksuniversiteit
Groningen, Groningen, The Netherlands
Suzanne Jacobsen (167), Faculty of Education, The University of British
Columbia, Vancouver, British Columbia V6T 1Z5, Canada
Sang Jin Kang (203), College of Education, Michigan State University, East
Lansing, Michigan 48824
Valerie E. Lee (225), School of Education, University of Michigan, Ann
Arbor, Michigan 48109
Marlaine E. Lockheed (131), The World Bank, International Bank for Re-
construction and Development, Washington, D. C. 20433
Nicholas T. Longford (115, 131), Research Statistics Group, Educational
Testing Service, Princeton, New Jersey 08541
Lindsay Paterson (13,85), Centre for Educational Sociology, University of
Edinburgh, Edinburgh EH8 9LW, Scotland
Ian Plewis (53), Thomas Coram Research Unit, Institute of Education,
University of London, London WC1N 1AZ, Great Britain
IX
X Contributors
David Raffe (149), Centre for Educational Sociology, University of
Edinburgh, Edinburgh EH8 9LW, Scotland
Stephen W. Raudenbush (1, 203), College of Education, Michigan State
University, East Lansing, Michigan 48824
Brian Rowan (203), College of Education, Michigan State University, East
Lansing, Michigan 48824
Julia B. Smith (225), School of Education, University of Michigan, Ann
Arbor, Michigan 48109
J. Douglas Willms (1), Centre for Policy Studies in Education, The Uni-
versity of British Columbia, Vancouver, British Columbia V6T 1Z5,
Canada, and Centre for Educational Sociology, University of
Edinburgh, Edinburgh E8H 9LW, Scotland
Ruth Zuzovsky (25), Department of Education, Tel Aviv University, Ram at
Aviv 69 978, Israel
Preface
An irony in the history of quantitative studies of schooling has been
the failure of researchers* analytic models to reflect adequately the social
organisation of life in classrooms and schools. The experiences that chil-
dren share within school settings and the effects of these experiences on
their development might be seen as the basic material of educational re-
search; yet, until recently, few studies have explicitly taken account of the
effects of the particular classrooms and schools in which students and
teachers share membership.
Researchers have been painfully aware of this shortcoming, because
for many years methodologists have criticised quantitative studies of
schooling for using statistical methods that fail to reflect the hierarchical,
social organisation of schooling. The difficulty has been, quite simply, that
adequate statistical methods have not been available. During the 1980s,
however, advances in statistical theory merged with thinking about re-
search on schooling to produce a new set of statistical approaches known
as "multilevel" or "hierarchical" linear models. Many papers, cited in the
chapters of this volume, have appeared in technical journals which con-
sider the statistical and computational issues that arise in applying these
models. In a recent volume entitled, Multilevel Analysis of Educational Data,
edited by R. D. Bock (1989), a number of the methodologists involved in
creating multilevel methods described the technical aspects of their ap-
plication.
The present volume marks an important transition in the history of
multilevel research, for it shows that ownership of these methods has
begun to pass from the methodologists to the broader research commu-
nity. Multilevel analyses are no longer restricted to technical journals
where they have appeared only to illustrate the methods. Instead, soci-
ologists, educational psychologists, and policy researchers have begun to
apply these methods to tackle a variety of research problems which would
have been difficult to address credibly with traditional methods. Indeed,
the book presents a sample of some of the best in the first wave of appli-
cations of multilevel analysis in education.
One purpose of the book is to make the methods available to a broader
audience than they have been heretofore. The introductory section com-
xi
xii Preface
prises two chapters: the first considers how the explicit modelling of the
organisational structure of schooling creates new opportunities for re-
search. The second chapter is a basic guide to the techniques of multilevel
modelling. It is written for those who have only a rudimentary knowledge
of statistics. We believe, however, that an effective and enjoyable way to
learn about this methodology is to see how researchers have used it in a
variety of ways. The remaining sections of the book provide substantive
applications of the methods to the study of curricular coverage and cur-
ricular reform in several countries (Chapters 3-5), the evaluation of the
performance of schools and entire educational systems (Chapters 6,8, 9,
and 10), an examination of the changing social distribution of achievement
in a society (Chapter 7), the evaluation of innovative educational programs
(Chapters 11 and 12), and the study of the organisational context of
teaching (Chapters 13-15). In contrast to other books on this topic, the
presentations are nontechnical; the emphasis is on substance rather than
method. But the authors have clarified the ulogic-in-use" of the approach,
and we think a broad range of researchers will, as a result, gain a new un-
derstanding of that logic.
The book also considers some of the likely benefits of using multilevel
methods to study schools and classrooms. One benefit is the increased
credibility of the statistical findings. The findings take into account the
dependence in the responses of the students sharing a classroom or
school; they allow the investigator to distinguish between variation at each
level (e.g., variation among schools and variation within schools); they
allow specification of each variable at the conceptually appropriate level;
and they allow relationships among variables to differ in different educa-
tional contexts. Perhaps more important, the methods allow us to ask
research questions which probably would otherwise have remained
unasked.
The volume resulted from an international conference held during the
summer of 1989 and sponsored by the Centre for Educational Sociology
at the University of Edinburgh. With the support of the Economic and
Social Research Council of the United Kingdom, the Centre sought to
stimulate interest in multilevel research by bringing together researchers
from around the world who were at the forefront in applying these new
methods to important educational research problems. The conference
generated among those attending a new sense of possibility for quantita-
tive research in education. Our hope is that this volume will effectively
communicate that spirit to a far broader audience.
Stephen Raudenbush
Doug Willms
Acknowledgments
Recently, statisticians have developed powerful statistical techniques
for analysing multilevel data. The Centre for Educational Sociology (CES),
University of Edinburgh, has been at the forefront in applying these
methods to educational data. It sponsored several instructional work-
shops on the use of the methods and had input into the design of several
international projects. In August 1989, with funding from the United
Kingdom Economic and Social Research Council (ESRC), the CES spon-
sored an international conference on the applications of multilevel
methods in educational research. The Centre invited contributions from
a number of researchers who were actively using these methods in their
own research. The purpose of the conference was not to examine technical
details pertaining to multilevel methods, but to ask: What can be learned
about educational processes that would not have been possible to discern
with conventional methods, and what new questions arise as a conse-
quence of employing a multilevel perspective? This book comprises the
papers presented at the conference.
We are grateful to the ESRC for funding the conference (ESRC Grant No.
XC00262087) and for their general support of methodological work per-
taining to survey methods and statistics in education. Since 1987 it has
supported the CES as a Designated Research Centre (ESRC Grant No.
C00280004). Its support has advanced the understanding of new meth-
odological techniques and their application to a range of substantive is-
sues, especially in the assessment of current educational reforms.
We also wish to acknowledge the contribution of the U.S. Spencer
Foundation. It previously provided support to Stephen Raudenbush and
Tony Bryk for the development of multilevel techniques. It also provided
research support to Doug Willms through a postdoctoral fellowship and
a small grant. Two of the authors, Valerie Lee and Adam Gamoran, have
recently been awarded Spencer Foundation postdoctoral fellowships to
continue work in this vein.
We are indebted to David Raffe, co-Director of CES, for his substantial
contribution to the organisation of the conference. Joan Hughes, Margaret
Burnett, Moira Burke, and Cathy Garner took care of the many details in
planning the conference; to them we also give our thanks.
xin
xiv Acknowledgments
We are grateful also to Marjorie Oyer for her help in preparing manu-
scripts. Martin Turner provided considerable assistance in the final pro-
duction of the book, including configuration of the hardware and
modification of the software to produce the numerous tables and equa-
tions found in the book. We are thankful to him for the many long after-
noons and late nights he spent on this work. Several others made specific
contributions in regard to particular chapters. These are acknowledged
separately in the first footnote of each chapter.
The opinions and conclusions expressed in the book are not neces-
sarily shared by the ESRC, the CES, or other organisations. Responsibility
for these rests with the authors.
Stephen Raudenbush
Doug Willms
SECTION I
INTRODUCTION
Chapter 1
The Organisation of Schooling and its Methdological Implications
Stephen W. Raudenbush1
Michigan State University
and
J. Douglas Willms
University of British Columbia
& University of Edinburgh
The Evaluation of Educational Reform
The attempt to study the effects of schooling and to use the resulting
information to improve schools constitutes a large-scale enterprise in many
countries. This enterprise takes a variety of forms in each country.
Government agencies monitor the effects of educational organisations with
the aim of identifying needed improvements. Evaluators design studies to
assess the effects of particular educational reforms. And academic
researchers examine how schools, classrooms, and teaching are structured,
how those structures change, and how those changes affect children's
development. The academic research may be labelled as basic rather than
applied, but the intended end product of the research endeavour, taken as
a whole, remains nonetheless the improved functioning of formal education.
This volume brings together recent experience from a number of
countries in producing empirical evidence useful for the improvement of the
conditions and outcomes of schooling. What these diverse studies share,
beyond this broad goal, is their careful attention to the hierarchical,
multilevel organisational structure within which formal education occurs.
In nearly all countries, children learn in classrooms situated within
schools which are regulated by policies made by districts or educational
authorities. The actions of the districts or educational authorities, in turn,
are constrained by laws and policies enacted at the state or provincial level
within a legal and political framework constructed at the national level. This
basic structure of governance of education systems has fundamental
consequences, both for conceptions of educational reform and for studies
of the processes and effects of schooling.
Attempts to reform schooling can be implemented at each of the
various levels, subject, of course, to constraints created by actors at other
1 We are grateful to Lindsay Paterson for comments on an earlier version of this
chapter.
Schools, Classrooms, and Pupils
Copyright © I99l by Academic Press, Inc. \
All rights of reproduction in any form reserved.
2 Stephen W. Raudenbush & J. Douglas Willms
levels, both above and below. Almost inevitably, however, the intended
outcomes of a reform, if they are to occur at all, will occur at a lower level
than that at which the reform is implemented. A auricular reform enacted
nationally will be irrelevant if it fails to influence the practice of school
principals and classroom teachers. A change in the sequencing and pacing
of instruction implemented within a classroom will deserve praise only if it
engages students more effectively, enables them to learn more, or increases
their chances of productive employment after graduation.
Conceptions of educational reform, then, must be either explicitly or
implicitly multilevel. A theory of school improvement might say (or imply)
that if we modify certain factors which vary at a higher level (e.g., principal
leadership, teacher supervision, distribution of auricular materials), a
process will be set in motion which will lead to effects at a lower level:
teachers will interact differently with children and children will learn more.
Unit of Analysis. One methodological consequence of this conception is
that, in many quantitative studies, the key independent variables will vary
at a higher level of aggregation than do the dependent variables. This fact
alone has posed serious problems for researchers, as is manifest by the
substantial literature on the choice of unit of analysis in educational
research reviewed, for example, by Haney (1980). This research poses the
question as follows: should the researcher use the student as the unit,
ignoring the differing organisational contexts in which education occurs and
is reformed? Or should the organisational context (e.g., the classroom or
school) be the unit of analysis, requiring the aggregation of student data
within each context, ignoring all variation among students within a given
context? Cronbach (1976), Burstein (1980) and others have recognized for
some time that the "choice of unit of analysis" is the wrong question because
the variation at each level is potentially of interest and ought not be
ignored.
Two additional methodological concerns emanate from the organisation-
al structure of schooling and its governance, and are therefore generic in
studies of schooling and school reform. In the statistical literature these are
termed the problems of confounding and interaction, but they take on
particular forms when the important variables vary at multiple levels of
aggregation.
Confounding. Whenever we study the effects of policy or practice on
teachers or students, we must account for influences over which policy-
makers and practitioners have little or no control. These influences
themselves vary at different levels of the educational hierarchy. In a study
of the association between secondary school organisation and student
achievement, for example, researchers must attempt to adjust for differences
in student demographic background, prior skill, and motivation, variables
defined at the student level. Ignoring such student differences would
typically bias estimates of the effect of school organisational variables, which
may be correlated with school differences in student background. Other
potentially confounding variables, more often ignored, are defined at the
school level and do not vary among students within a school. For example,
schools may differ in their local labour market conditions and in the
socioeconomic levels of the communities in which they are located. These
contextual conditions may be related both to school organisation and to