Conducting Quantitative Research In Education

206 Pages·2020·18.841 MB·English

by  Saiyidi Mat Roni, Margaret Kristin Merga, Julia Elizabeth Morris

Saiyidi Mat Roni 
Margaret Kristin Merga 
Julia Elizabeth  Morris 
Conducting
Quantitative
Research in
Education
Conducting Quantitative Research in Education
Saiyidi Mat Roni • Margaret Kristin Merga
Julia Elizabeth Morris
Conducting Quantitative
Research in Education
Saiyidi Mat Roni Margaret Kristin Merga
School of Business and Law School of Education
Edith Cowan University Edith Cowan University
Joondalup, WA, Australia Perth, Australia
Julia Elizabeth Morris
School of Education
Edith Cowan University
Mount Lawley, WA, Australia
ISBN 978-981-13-9131-6 ISBN 978-981-13-9132-3 (eBook)
https://doi.org/10.1007/978-981-13-9132-3
© Springer Nature Singapore Pte Ltd. 2020
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology
now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
The publisher, the authors, and the editors are safe to assume that the advice and information in this book
are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the
editors give a warranty, express or implied, with respect to the material contained herein or for any errors
or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims
in published maps and institutional affiliations.
This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd.
The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721,
Singapore
Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 What This Book Will Do . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 The Importance of Engaging with Quantitative Research
Methods and Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 The Neglected Non-parametric Analysis . . . . . . . . . . . . . . . . . . . . . 3
1.4 Format and Suggested Engagement . . . . . . . . . . . . . . . . . . . . . . . . . 4
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2 Getting Started: What, Where, Why . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2 What Are Quantitative Methods All About and When Should
I Use Them? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2.1 The Research Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2.2 When Are Quantitative Methods Appropriate? . . . . . . . . . . 10
2.2.3 What Types of Quantitative Measures
Are Available to Me? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2.4 In the Event of Writing Your Own Survey . . . . . . . . . . . . . . 12
2.3 Where Can I Collect Quantitative Data and How
Do I Go About It? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.3.1 What Could My Setting and Sample Look Like? . . . . . . . . 14
2.3.2 Dissemination Methods for Surveys and Other
Quantitative Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.3.3 How Many Times Should I Collect Data? . . . . . . . . . . . . . . 18
2.4 Why Use Quantitative Methods? A Few Examples . . . . . . . . . . . . . 18
2.4.1 Pure Quantitative Research Designs . . . . . . . . . . . . . . . . . . 18
2.4.2 Mixed Methods Research Designs . . . . . . . . . . . . . . . . . . . . 21
2.5 Final Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3 Conducting Research with Children and Students . . . . . . . . . . . . . . . 25
3.1 Age Appropriate Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.2 Satisficing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
v
vi Contents
3.2.1 This Is Not a Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.2.2 I Want to Know What You Really Think, Not What
You Think I Want You to Say . . . . . . . . . . . . . . . . . . . . . . . . 28
3.2.3 No One Will Know What You Said . . . . . . . . . . . . . . . . . . . 28
3.2.4 I Value Your Opinion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.3 The Importance of Piloting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.3.1 Pre-pilot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.3.2 Pilot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.4 What Am I Looking for in a Pilot? . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.4.1 You Don’t Know What You Don’t Know . . . . . . . . . . . . . . . 30
3.4.2 You Want Your Data to Actually Be Reliable . . . . . . . . . . . 31
3.4.3 Your Reviewers Will Appreciate It . . . . . . . . . . . . . . . . . . . 32
3.5 Ethics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.6 Understanding and Engaging Schools . . . . . . . . . . . . . . . . . . . . . . . 34
3.6.1 The Cover Letter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.6.2 Other Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.7 Final Comment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4 Data Types and Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.1 Data Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.2 Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
5 Data Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
5.1 Data Entry in SPSS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
5.2 Knowing Your Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
5.3 Recoding and Recomputing Data . . . . . . . . . . . . . . . . . . . . . . . . . . 59
6 Analysis: Difference Between Groups . . . . . . . . . . . . . . . . . . . . . . . . . . 65
6.1 Mann-Whitney U . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
6.1.1 Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
6.1.2 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
6.1.3 Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
6.1.4 Checking the Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . 71
6.1.5 Descriptive Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
6.1.6 Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
6.2 Kruskal-Wallis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
6.2.1 Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
6.2.2 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
6.2.3 Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
6.2.4 Checking the Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . 83
6.2.5 Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
6.3 Chi-square . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
6.3.1 Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
6.3.2 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
Contents vii
6.3.3 Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
6.3.4 Checking the Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . 89
6.3.5 Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
6.4 McNemar test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
6.4.1 Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
6.4.2 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
6.4.3 Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
6.4.4 Checking the assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . 95
6.4.5 Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
6.5 Cochran’s Q test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
6.5.1 Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
6.5.2 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
6.5.3 Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
6.5.4 Checking the Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . 101
6.5.5 Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
6.6 Wilcoxon Signed-Rank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
6.6.1 Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
6.6.2 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
6.6.3 Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
6.6.4 Checking the Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . 106
6.6.5 Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
7 Analysis: Correlation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
7.1 Spearman’s Rho . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
7.1.1 Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
7.1.2 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
7.1.3 Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
7.1.4 Checking the Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . 115
7.1.5 Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
7.2 Kendall’s Tau . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
7.2.1 Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
7.2.2 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
7.2.3 Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
7.2.4 Checking the Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . 124
7.2.5 Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
7.3 Cramer’s V . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
7.3.1 Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
7.3.2 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
7.3.3 Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
7.3.4 Checking the Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . 131
7.3.5 Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
8 Analysis: Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
8.1 Simple Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
8.1.1 Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
8.1.2 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
viii Contents
8.1.3 Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
8.1.4 Checking the Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . 144
8.1.5 Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
8.2 Binomial Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
8.2.1 Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
8.2.2 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
8.2.3 Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
8.2.4 Checking the Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . 160
8.2.5 Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
8.3 Multinomial Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
8.3.1 Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
8.3.2 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
8.3.3 Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
8.3.4 Checking the Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . 171
8.3.5 Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
9 Write Up and Research Translation . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
9.1 The Plain English Report for Schools . . . . . . . . . . . . . . . . . . . . . . . 178
9.2 The Journal Article . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
9.2.1 Good Fit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
9.2.2 Reputable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
9.2.3 Fast . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
9.3 Unique Challenges in Publishing Quantitative
and Mixed-Methods Articles in education . . . . . . . . . . . . . . . . . . . . 182
9.4 Thesis by Publication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
9.4.1 But Is This Right for Me? . . . . . . . . . . . . . . . . . . . . . . . . . . 186
9.5 Plain English Dissemination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
9.6 Altmetrics and (Social) Media-Supported Dissemination . . . . . . . . 189
9.7 The Conference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
9.8 Final Comment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
10 Conclusion and Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
10.1 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
10.1.1 Readings About Mixed Methods Design . . . . . . . . . . . . . . 195
10.1.2 Readings About Experimental Methods Design . . . . . . . . 195
10.1.3 Readings About Sample Size, Power
and Effect size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
10.1.4 Readings About Ethical Issues in Education
and Social Science Research . . . . . . . . . . . . . . . . . . . . . . . 196
10.1.5 Readings About Survey Design . . . . . . . . . . . . . . . . . . . . . 196
10.1.6 Readings About Validity and Reliability . . . . . . . . . . . . . . 197
10.1.7 Readings About Quantitative Analyses . . . . . . . . . . . . . . . 197
10.1.8 Readings About Reporting Educational Research . . . . . . . 197
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
Chapter 1
Introduction
Welcome to Conducting Quantitative Research in Education! This book has been
written to help support higher degree by research students, early career academics
and life-long researchers who are looking to increase their capacity to both choose
and use quantitative data collection and analysis in educational research.
1.1 What This Book Will Do
By the end of this book, you should be able to design and undertake a research proj-
ect which is entirely or partly reliant on quantitative research methods. You will
have an understanding of some of the unique contextual factors involved in conduct-
ing quantitative research in educational settings, and you will be able to prepare
your data for a broad range of analyses. We will walk you step-by-step through each
of these stages, so even those of you who are absolutely new to quantitative design
and data analysis (and slightly freaked out by the weird symbols and things) should
be able to apply the ideas that we cover herein to your own research plans with a
degree of confidence. Each of these analytical chapters will be self-contained,
beginning with its own overview of a statistical procedure, a survey instrument,
research question, hypothesis, SPSS procedures and output.
Unlike many statistical support books, we consistently use a similar set of vari-
ables to illustrate the analyses in this book. We do this so you may focus your atten-
tion on understanding each analysis, as opposed to deciphering the purpose of
different variables in different research designs. This approach also allows you to
see multiple analyses conducted on the same variables, which helps to minimise
confirmation bias where researchers unknowingly bias toward asserting their
hypotheses while disregarding other possible explanations. In this book we use
reading frequency as a dependent variable in multiple chapters. In doing so, you can
see how reading frequency is influenced by gender, encouragement to read and
other factors, and how we conducted analyses to justify conclusions.
© Springer Nature Singapore Pte Ltd. 2020 1
S. Mat Roni et al., Conducting Quantitative Research in Education,
https://doi.org/10.1007/978-981-13-9132-3_1
2 1 Introduction
We also include examples of how the results from each analysis could be
reported, based on APA style, as this is most typically used in education. In addition,
we provide advice about how to write up your work for publication, and highlight
additional readings that you can draw upon if you find yourself wishing to delve
more deeply into non-parametric statistics. You should be able to use this book to
support educational research with samples derived from pre-school, primary, sec-
ondary and tertiary education contexts.
1.2 The Importance of Engaging with Quantitative Research
Methods and Analysis
Quantitative data analysis occupies an odd place in educational research. While
there are many exceptional and well-known researchers working in this paradigm in
education, and it is prevalent in some sub-fields such as educational psychology,
we’ve noticed that research involving quantitative analysis is less common in other
sub-fields. Some supervisors in education may not feel comfortable supporting stu-
dents wishing to work with this approach due to lack of experience and knowledge
with this method. We have even detected distinct nose-wrinkling at the mention of
quantitative data analysis in some educational research contexts. Perhaps this is
because the research questions we seek to explore, and the theories that underpin
our research in education, are often better suited to a qualitative method. It could
also be due to the tension between adherents of qualitative and quantitative
approaches, which situates them as approaches in opposition. However, researchers
in educational and social sciences are increasingly moving toward accepting a
“legitimate complementarity of paradigms” (Salomon 1991, p. 10), with both para-
digms seen as important and valuable when used in a way that is responsive to both
the intended inquiry and context.
For the contemporary researcher, we need to get over this unnecessary dichot-
omy. It is useful to be knowledgeable across the broadest range of methods possible,
including quantitative methods, so that we can understand and interpret the litera-
ture that we review in our research areas. Even if the body of your current work uses
qualitative methods, understanding of quantitative methods adds the option of going
down this path for parallel or follow-up data collection on current research projects,
enabling the interrogation of a research concern through multiple phases and the
generalisation of exploratory research (Creswell and Plano Clark 2011). Knowledge
of this area can also help us to support and/or supervise higher degree by research
students wishing to use this approach. Having at least a basic understanding of how
to employ a range of methods can enable you to take a creative and novel position
on a research area. For example, where you have already done a great deal of quali-
tative research, you may now seek to create a survey tool based on some of the
emerging themes to test their generalisabilty.
There are a range of advantages to using a quantitative approach. Quantitative
approaches are sometimes related to higher impact, both in terms of publication and
translation, perhaps due to their broader capacity for generalisability. While many