Primary school achievement gaps and school decisions to support the academic achievement of ...

386 Pages·2016·4.82 MB·English

Faculty of Education and Social Work

Primary school achievement gaps and school
decisions to support the academic
achievement of disadvantaged students with
data: A cross-country comparative study
Nicole Bien

A thesis submitted in fulfilment of the requirements
for the degree of Doctor of Philosophy
Faculty of Education and Social Work

2016
ii

Faculty of Education and Social Work
Office of Doctoral Studies

AUTHOR’S DECLARATION

This is to certify that:

I. this thesis comprises only my original work towards the Doctor of
Philosophy Degree

II. due acknowledgement has been made in the text to all other material used

III. the thesis does not exceed the word length for this degree.

IV. no part of this work has been used for the award of another degree.

V. this thesis meets the University of Sydney’s Human Research Ethics
Committee (HREC) requirements for the conduct of research.

Signature:

Name: Nicole Ma Bien
Date: 29 March 2016
iii

Acknowledgements
The journey to undertake and complete this thesis has been incredibly rewarding. I
am deeply grateful to the many people who accompanied me on my journey and
supported me at various junctures so that I could reach my destination. First and
foremost, I wish to express my heartfelt gratitude to my Doctoral Supervisor, Dr.
Ilektra Spandagou for her generosity, patience and encouragement through the
entire course of my candidature. I have benefited immensely from her tireless
guidance and academic insights. I also wish to thank my Associate Supervisor,
Associate Professor David Evans for his review and constructive feedback.

I owe my greatest thanks to my family for their love and unwavering support. Bill, my
partner and soulmate, encouraged me to undertake this doctoral study and was my
biggest cheerleader through the entire journey. More importantly, Bill’s steadfast
support for my career change has been indispensable. My sisters, Dr. Ma and Dr. Ma
provided valuable thesis-writing advice and my brothers, Ken and Mitch offered
ongoing moral support. My mother, Jenny, taught me perseverance through her own
life story. Finally, my children, Bandon and Kaela, who understood the importance of
this journey by checking my progress on a regular basis.

My thanks also go to Dr. Alun Pope for his review and feedback of the quantitative
work and Sonia Bartoluzzi for proof-reading the final copy of this thesis. I am
particularly thankful to the school administrators and teachers who gave up their
precious time to participate in the interviews. The University also made the thesis
possible by awarding generous funding support through the Thomas and Ethel Mary
Ewing Scholarship, the Alexander Mackie Research Scholarship, and the
Postgraduate Research Support Scheme. Finally, I am grateful to Dr. Armistead and
Carol Mille, leaders of the School Board where I worked for approving a sabbatical
for me to complete this thesis.
iv

Abstract
Reducing the academic disadvantage of all students is a significant educational goal
for many countries. Increasingly, education reforms around the world, including
those in Australia and the United States have sought to reduce achievement gaps by
adopting a strategy of embedding accountability anchored by standardised
assessment. Whether to meet federal and state educational requirements, to
provide transparency to the general public, or to inform curriculum and instruction
at individual schools, policy makers rely on assessment data and data-driven practice
to make a difference. Although external forces such as policy expectations are
generally the first step in creating social change, the internal beliefs of change agents
can impact their course of action. Being agents of change, school educators can
choose to adopt data-driven practice for compliance, or also to engage with data for
continuous improvement. Applying the efficacy theory and the theory of planned
behaviour from the social cognitive tradition, this thesis examined educators’ belief
mechanism regarding embracing data-practice, and considered the direct and
indirect benefits of data-engagement, as well as the costs that ensued for teaching
and learning.

Using standardised assessment results from 2008–2013 in Australia and two
counties in California, and six case studies across New South Wales, California, and
Hawaii, the present mixed methods research found evidence of progress in raising
the proficiency of disadvantaged students, but not in narrowing achievement gaps
between advantaged and disadvantaged students. The case studies suggest a
positive relationship between academic proficiency progress and data engagement.
This can be explained by the structural design and operational procedures of the
data-driven process enhancing educators’ attitudes, intention, perceived efficacy
beliefs, and perceived behavioural control relating to the challenging task of raising
the educational outcomes of disadvantaged students. As a result, participants could
see beyond mere compliance with data-driven practice to its potential for
professional improvement.
v

Contents
AUTHOR’S DECLARATION .......................................................................................ii
Acknowledgements .............................................................................................. iii
Abstract ................................................................................................................ iv
List of Tables ......................................................................................................... ix
List of Figures ........................................................................................................ xi
Definition of Key Terms ........................................................................................ xii
General Terminology .................................................................................................. xii
American Terminology .............................................................................................. xiii
Australian Terminology ............................................................................................. xiv
Chapter 1 Introduction .......................................................................................... 1
Context……. ................................................................................................................. 1
Rationale for the Study ................................................................................................ 5
Theoretical Framework .............................................................................................. 11
Self-efficacy and collective-efficacy theory. ................................................................... 12
Theory of planned behaviour (TPB). ............................................................................... 14
A combined construct. .................................................................................................... 15
Purpose of the Study .................................................................................................. 16
Research Questions ................................................................................................... 18
Significance of the Study ............................................................................................ 18
Overview of the Methods ........................................................................................... 20
Overview of the Chapters ........................................................................................... 21
Chapter 2 Contextual Framework: the Evolution of Education Transparency and
Accountability ...................................................................................................... 23
Policy Context ............................................................................................................ 24
The Race to Produce 21st Century Knowledge Workers ................................................ 26
Education Inequity and the Persistent Achievement Gap ............................................ 28
Education inequity in the US .......................................................................................... 30
Education inequity in Australia. ...................................................................................... 33
Federal Solutions: Education Accountability ............................................................... 37
The rise and fall of No Child Left Behind. ....................................................................... 38
The Melbourne Declaration and its implementation. .................................................... 40
The Quest for Transparency and the Outsized Role of Testing ..................................... 45
Conclusion.. ............................................................................................................... 48
Chapter 3 The Impact of Test-Based Accountability Reforms ................................ 50
Needs and Challenges of Disadvantaged Students ...................................................... 51
Equal Access to External Assessment – Goal or Reality? .............................................. 53
Effects on Student Performance and the Achievement Gap ......................................... 60
Academic outcomes of disadvantaged students in the US ............................................ 62
Academic outcomes of disadvantaged students in Australia. ........................................ 67
Unintended Consequences of Test-Based Accountability ............................................. 71
vi

Cheating to inflate school performance. ........................................................................ 72
Teaching to the test and narrowing the curriculum. ...................................................... 72
‘Education triage’. ........................................................................................................... 73
Parallel Evidence in Australia...................................................................................... 75
Contradictory Findings ............................................................................................... 77
The Promise of Data-Driven Practice and the Reality of Practice .................................. 79
Perceived trust and benefits in external data. ............................................................... 81
The nature of data use. ................................................................................................... 84
Data-engagement criteria – human factors. .................................................................. 84
Data-engagement criteria – systemic factors. ................................................................ 86
Conclusion.. ............................................................................................................... 89
Chapter 4 Theoretical Framework ........................................................................ 92
Identifying a Theoretical Framework .......................................................................... 92
Efficacy Theory – An Overview ................................................................................... 94
Mastery experience. ....................................................................................................... 96
Vicarious experience. ...................................................................................................... 97
Social persuasion. ........................................................................................................... 97
Physiological and emotional state. ................................................................................. 98
Efficacy in the education context. .................................................................................. 99
Justifications for the efficacy construct. ....................................................................... 103
The Theory of Planned Behaviour ............................................................................. 105
TPB in the education context........................................................................................ 109
Justifications for the Planned Behaviour Theory. ......................................................... 110
Theoretical Issues .................................................................................................... 111
Justifications for Combining the Two Theories .......................................................... 112
Conclusion.. ............................................................................................................. 115
Chapter 5 Methodology ..................................................................................... 116
Context of the Study ................................................................................................ 116
Philosophical Assumption and Implication for Research ............................................ 118
Mixed Methods Research Paradigm ......................................................................... 119
Rationale for mixed methods research. ....................................................................... 120
Research Questions ................................................................................................. 123
Research Design ....................................................................................................... 124
The Quantitative Study ............................................................................................ 126
Data sources. ................................................................................................................ 128
Sample. ......................................................................................................................... 130
Definition of progress. .................................................................................................. 131
Definition of participation. ........................................................................................... 132
Data analysis and procedure. ....................................................................................... 133
Limitations. ................................................................................................................... 142
The Qualitative Study ............................................................................................... 144
The role of the researcher. ........................................................................................... 146
Ethical considerations. .................................................................................................. 147
Sampling procedure. ..................................................................................................... 148
Participating schools and participants. ......................................................................... 151
Data collection and sources of data. ............................................................................ 159
Data analysis. ................................................................................................................ 166
Validity and reliability. .................................................................................................. 169
vii

Limitations. ................................................................................................................... 169
Conclusion.. ............................................................................................................. 170
Chapter 6 Quantitative Findings: Academic Progress in California ....................... 172
Data and Statistical Procedures ................................................................................ 173
Statistical procedures. .................................................................................................. 173
Data summary. .............................................................................................................. 175
Reading Achievement Trends ................................................................................... 177
Mathematics Achievement Trends ........................................................................... 183
Progress on Achievement Gap .................................................................................. 189
Reading. ........................................................................................................................ 189
Mathematics. ................................................................................................................ 192
Inclusion in Assessment ........................................................................................... 195
Discussion… ............................................................................................................. 195
Summary…….. .......................................................................................................... 198
Chapter 7 Quantitative Findings: Academic Progress in Australia ........................ 200
Data and Statistical Procedures ................................................................................ 200
Data summary. .............................................................................................................. 201
Statistical analysis. ........................................................................................................ 202
Reading and Numeracy Achievement Trends ............................................................ 205
Subgroup differences in reading and numeracy. ......................................................... 208
Progress on NAPLAN. .................................................................................................... 213
Progress on Achievement Gap. ................................................................................. 217
Trends on NAPLAN Absence and Withdrawal ............................................................ 219
Discussion… ............................................................................................................. 222
Summary…… ............................................................................................................ 225
Chapter 8 Becoming Data-Driven Schools ........................................................... 227
Case Context, Data Definition, and Student Support Structure .................................. 228
Case context. ................................................................................................................ 228
Data definition. ............................................................................................................. 243
Support structure for low-performing students. .......................................................... 245
The Evolution of Data Engagement ........................................................................... 249
Perception of data utility. ............................................................................................. 252
Nature of data use. ....................................................................................................... 256
Similarities and Differences ...................................................................................... 265
Instructional leadership. ............................................................................................... 265
Goal-oriented, accountable and transparent. .............................................................. 266
Goal attainment – the currency for programming and instructional freedom. ........... 266
Not 100% on board. ...................................................................................................... 268
Strategic versus tactical orientation. ............................................................................ 269
Perceived versus actual accountability pressure. ......................................................... 270
Portfolio view versus strict assessment view. .............................................................. 271
Conclusion.. ............................................................................................................. 272
Chapter 9 Interpreting Behavioural Change ........................................................ 274
The Theoretical Construct ........................................................................................ 275
The Formation and Influence of Efficacy Belief, Intention and Behaviour Control ....... 277
Higher degree of perceived behavioural control and mastery experience. ................. 278
viii

Positive vicarious experience........................................................................................ 283
An improved physiological and emotional state. ......................................................... 285
Affirming social persuasion........................................................................................... 287
Influences of attitude and subjective norms. ............................................................... 290
Relevance and Implications of Strong Efficacy Belief and Planned Behaviour ............. 291
Students are not the problem regardless of background. ........................................... 293
Student-centric practice. .............................................................................................. 294
Collaboration over isolation. ........................................................................................ 296
Subsequent direct and indirect benefits. ..................................................................... 299
Costs of Data Engagement ........................................................................................ 306
Constant testing. ........................................................................................................... 306
Seeing the forest for the trees. ..................................................................................... 307
Compromising creativity. .............................................................................................. 308
Limited visibility of students not at risk of failing. ........................................................ 309
Conclusion.. ............................................................................................................. 310
Chapter 10 Discussion and Conclusions .............................................................. 313
Context…….. ............................................................................................................ 313
Significance of the Study .......................................................................................... 313
Key Findings............................................................................................................. 314
Quantitative findings. ................................................................................................... 314
Qualitative findings. ...................................................................................................... 318
Implications of the Study ......................................................................................... 322
Prospects for closing the achievement gaps. ............................................................... 322
The ‘real’ value of data-driven practice. ....................................................................... 325
Lessons from policy-borrowing and policy-lending. ..................................................... 327
Limitations and Further Research Direction .............................................................. 330
Conclusion…………….. ................................................................................................ 331
References ......................................................................................................... 334
Appendices ........................................................................................................ 359
Appendix A Recruitment Letter ........................................................................................ 359
Appendix B Participant Information Statement................................................................ 360
Appendix C Participant Consent Form .............................................................................. 362
Appendix D Interview Guide ............................................................................................. 364
Appendix E Colour-code Worksheet of Student Achievement at Almond ....................... 366
Appendix F Pacing Guide at Kukui ..................................................................................... 367
Appendix G Data Team Process at Hibiscus ...................................................................... 368
Appendix H Learning Community Meeting Protocol at Hibiscus ...................................... 369
Appendix I Instructional Leadership Team Meeting Agenda at Kukui .............................. 370
Appendix J Evaluation of Student Performance and Action Plan Part I at Almond .......... 371
Appendix K Evaluation of Student Performance and Action Plan Part II at Almond ........ 372
ix

List of Tables
Table 2.1 Government Recurring School Funding ..................................................... 32
Table 2.2 Enrolment in Australian Schools (2008) ..................................................... 35
Table 3.1 Score Changes on the NAEP (1990 and 2013) ........................................... 63
Table 3.2 NAPLAN Score and Minimum Proficiency Percentage Change between
2008 and 2014 .......................................................................................... 68
Table 5.1 Advantaged and Disadvantaged Subgroups for California Data
Comparison ............................................................................................. 135
Table 5.2 Standardised Beta Coefficients of Australian School Background Variables
................................................................................................................ 139
Table 5.3 Disadvantaged and Advantaged Proxies for NAPLAN Categorical Variables
................................................................................................................ 140
Table 5.4 Categorisation of NAPLAN Numeric Variables ......................................... 140
Table 5.5 Summary of Case Study Participant ......................................................... 152
Table 5.6 Student Profiles at California Participating Schools and Districts (2011–
2012) ....................................................................................................... 154
Table 5.7 Staff Information at Almond and Walnut (2011–2012) .......................... 155
Table 5.8 Student Profiles at Hawaiian Participating Schools and Complexes (2011–
2012) ....................................................................................................... 156
Table 5.9 Staff Information at Kukui and Hibiscus (2011–2012) ............................. 156
Table 5.10 Australian Participating School Profiles (2012) ....................................... 158
Table 5.11 NAPLAN Mean Scale Scores (2012) .......................................................... 159
Table 6.1 Percentage of Schools Represented by Subgroup in Grade 3 .................. 176
Table 6.2 Percentage of Schools Represented by Subgroup in Grade 5 .................. 176
Table 6.3 Grade 3 Reading Means and Proficiency Rates at Almond County ......... 178
Table 6.4 Grade 5 Reading Means and Proficiency Rates at Almond County ......... 178
Table 6.5 Grade 3 Reading Means and Proficiency Rates at Walnut County .......... 181
Table 6.6 Grade 5 Reading Means and Proficiency Rates at Walnut County .......... 181
Table 6.7 Grade 3 Mathematics Means and Proficiency Rates at Almond County . 183
Table 6.8 Grade 5 Mathematics Means and Proficiency Rates at Almond County . 184
Table 6.9 Grade 3 Mathematics Means and Proficiency Rates at Walnut County . 185
Table 6.10 Grade 5 Mathematics Means and Proficiency Rates at Walnut County . 185
Table 6.11 Grade 3 Reading Achievement Gaps between Groups at Almond County
................................................................................................................ 190
Table 6.12 Grade 5 Reading Achievement Gaps between Groups at Almond County
................................................................................................................ 190
Table 6.13 Grade 3 Reading Achievement Gaps between Groups at Walnut County
................................................................................................................ 191
Table 6.14 Grade 5 Reading Achievement Gaps between Groups at Walnut County
................................................................................................................ 191
x

Table 6.15 Grade 3 Mathematics Achievement Gaps between Groups at Almond
County ..................................................................................................... 192
Table 6.16 Grade 5 Mathematics Achievement Gaps between Groups at Almond
County ..................................................................................................... 193
Table 6.17 Grade 3 Mathematics Achievement Gaps between Groups at Walnut
County ..................................................................................................... 194
Table 6.18 Grade 5 Mathematics Achievement Gaps between Groups at Walnut
County……………………………………………………………………………………………….194
Table 7.1 Disadvantaged and Advantaged Proxies for NAPLAN Variables ............. 202
Table 7.2 Average NAPLAN Scale Score Means and Standard Deviations (2008–
2013) ....................................................................................................... 209
Table 7.3 Average NAPLAN Proficiency Rates and Standard Deviations (2008– 2013)
................................................................................................................ 212
Table 7.4 Difference in NAPLAN Means (2013 less 2008) ....................................... 216
Table 7.5 Average NAPLAN Score Gaps and Growth Trends (2008–2013) ............. 217
Table 7.6 Average NAPLAN Proficiency Gaps and Growth Trends (2008–2013)..... 218
Table 7.7 NAPLAN Absentee/Withdrawal Percentage Change Over Time (2013 less
2008) ....................................................................................................... 221
Table 8.1 The Meaning of 'Data' in the Interviews………………………………………………244
Table 8.2 Nature of Data Use .................................................................................. 256