effects of test anxiety, time pressure, ability and gender

255 Pages·2004·5.16 MB·English

by  Jing Chen

EFFECTS OF TEST ANXIETY, TIME PRESSURE, ABILITY
AND GENDER ON RESPONSE ABERRANCE

DISSERTATION

Presented in Partial Fulfillment of the Requirements for
the Degree Doctor of Philosophy in the Graduate
School of The Ohio State University

By

Jing Chen, M.A., M.S.

*****

The Ohio State University
2004

Dissertation Committee:
Approved by
Professor Ayres D’Costa, Adviser

Professor Michael Browne --------------------------------
Adviser
Professor William E. Loadman College of Education
ABSTRACT

This research investigated the effects of test anxiety, time pressure, ability and
gender (independent variables), as well as their potential interaction effects on response
aberrance, as measured by W*, B*, INFIT, OUTFIT, X2 , X2 and M (dependent
dif ord
variables). In addition, the construct(s) underlying these 7 indices and the overlap in their
aberrance detection were explored. Finally, 4 types of test-taking behavior associated
with test anxiety and time pressure, namely, start-up anxiety, random guessing, early
quitting and plodding, were examined with respect to the independent variables. The
sensitivity of the 7 indices to the 4 types of test-taking behavior was demonstrated using a
computer simulation study beforehand.
The research data came from an English language test administered to 450
junior-high-school students in China. Half of the students received the test under the
timed condition, and the other half under the untimed condition.
For 5 of the 7 indices, ability was found to interact with the time condition with
regard to response aberrance. Time pressure tends to impact medium- and high-ability
students, but not low-ability students, on their response aberrance as measured by INFIT,
OUTFIT and X2 . For the within-ability component of response aberrance, students at
ord
all 3 ability levels exhibit an increase under time pressure, with low-ability students
ii
having the biggest increase. For the beyond-ability component of aberrance, students at
low and medium ability levels exhibit a decrease under time pressure, while students at
the high ability level show an increase. As expected, males consistently demonstrate
higher aberrance levels than females, as measured by W*, INFIT, OUTFIT and M
indices.
Construct analyses for the timed group identified a general aberrance factor
measuring deviance from the Guttman model; whereas, for the untimed group, a more
specific beyond-ability aberrance factor was identified. The observed overlap in
aberrance detection supports this finding. Normed against the Guttman model, INFIT,
OUTFIT and M are indicators of general aberrance, while W*, B* and X2 are
dif
indicators of more specific types of aberrance. In contrast, X2 measures aberrance
ord
from yet another perspective different from the Guttman model.
A self-report Test-taking Behavior Survey helps understand these findings.
Further research needs are suggested.
iii
Dedicated
to my parents Chen, Zhengming and Zhu, Daozhen,
and
to my husband Wang, Shaoyong
iv
ACKNOWLEDGMENTS

I wish to thank my adviser, Dr. Ayres D’Costa, for his intellectual guidance and
consistent encouragement throughout my doctoral program and dissertation stage, and for
his enthusiasm in my research topic and great patience in correcting my errors.
I also wish to thank Dr. Michael Browne and Dr. William Loadman for their
stimulating advice, valuable comments and encouragement.
I am grateful to the students who participated in this research, and the principals
and teachers who were involved in data collection, especially Jianchun Zhang, Li Zeng,
and Xianmin Zeng.
In addition, I would like to extend my thanks to Dr. Wilco Emons at Tilburg
University, The Netherlands, for his friendship and taking so much time discussing with
me about this research, pointing out critical issues, helping me with computer
programming, and giving pertinent and enlightening suggestions.
Finally, I wish to thank my parents and brother for their continuous
encouragement during the past six years of my study abroad and their tremendous help
with data collection and data entry. I also wish to express my gratitude to my husband
for helping me with computer programming in my research, for taking care of me and
whole-heartedly supporting me with great patience.
Without all those above, this dissertation would not have been possible.
v
VITA

April 10, 1972 ………………………... Born – Hubei, P.R. China
1993 ………………………………...… B.A., English for Science and Technology
Xi’an Jiaotong University, Xi’an, China

1996 …………………………………... M.A., Applied Linguistics
Xi’an Jiaotong University, Xi’an, China

1996-1998 ……………………………. Lecturer, Department of English
Xi’an Jiaotong University, Xi’an, China

1998-1999 …………………………… Graduate Fellow
The Ohio State University

1999- present ………………………….Graduate Research and Administrative Associate
The Ohio State University
2001 ………………………………….. M.A.S., Applied Statistics
The Ohio State University

FIELDS OF STUDY
Major Field: Education
Quantitative Research, Evaluation and Measurement in Education

vi
TABLE OF CONTENTS

Page
Abstract ……………………………………………………………………………… ii
Dedication .………………………………………………………………………….. iv
Acknowledgments ………………………………………………………………….… v
Vita ………………………………………………………………………………….. vi
List of Tables.……………………………………………………………………….. x
List of Figures.…………………………………………………………………….…. xiii

Chapters

1. Introduction ……………………………………………………………………. 1

1.1 Background ………………………………………………………………… 3
1.2 Purposes of the Research …………………………………………………... 7
1.3 Importance of the Research ………………………………………………… 9
1.4 Basic Assumptions …………………………………………………………. 13
1.5 Definitions of Terms ……………………………………………………….. 14

2. Review of the Literature ……………………………………………………….. 23

2.1 Effects of Test Anxiety on Test Performance ……………………………… 23
2.2 Effects of Time Pressure on Test Characteristics and Performance.……….. 28
2.3 Aberrant Response Patterns ……………………………………….……….. 35
2.4 Response Aberrance Assessment ………………………………………….. 38
2.4.1 General Review of Aberrance Indices ……………………………. 39
2.4.2 Aberrance Indices Under Study and Related Research …………… 43
2.4.3 Test Anxiety and Response Aberrance …………………………… 46
2.4.4 Gender Difference on Response Aberrance ………………………. 47
2.4.5 Moderators on “Anxiety/Response Aberrance” Relationship …….. 48
2.5 Summary …………………………………………………………………… 49

vii
3. Design and Method ……………………………………………………………. 50

3.1 Theoretical Rationale ………………………………………………………. 52
3.1.1 Selection of Independent Variables ………………………………. 52
3.1.2 Selection of Aberrance Indices …………………………………… 56
3.1.2.1 Aberrant Response Patterns Related to Test Anxiety
In General…………………………………………….. 57
3.1.2.2 Effects of Time Pressure on Test-Anxiety Associated
Aberrant Patterns …………………………………... 61
3.1.2.3 Diagnostic Ability of Selected Aberrance Indices …… 63
3.1.2.4 Validating the Usefulness of Selected Indices –
A Simulation Study …………………………………. 75
3.2 Population and Sample ……………………………………………………... 81
3.3 Instrumentation …………………………………………………………….. 84
3.4 Design ……………………………………………………………………… 93
3.5 Data Collection ……………………………………………………………... 96
3.6 Statistical Analyses ………………………………………………………… 99

4. Results …………………………………………………………………………. 114

4.1 Reliability and Validity of Three Instruments ……………………………... 115
4.2 Results of Study 1…………………………………………………………... 121
4.2.1 Calculation of DVs (7 Aberrance Indices) ……………………….. 121
4.2.2 Description of the sample ………………………………………….123
4.2.3 Interaction Effects and Main Effects of IVs………………………. 136
4.3 Results of Study 2…………………………………………………………... 161
4.3.1 Factor Analysis to Explore the Interrelationships Among 7 DVs……….. 161
4.3.2 Index Sensitivity……………………………………………………169
4.4 Results of Study 3………………………………………………….……….. 175
4.4.1 Differences in Self-Reported Test Taking Strategies………………175
4.4.2 Relationship of Aberrance Indices to Test-taking Strategies……… 181
4.5 Summary……………………………………………………………………. 186

viii
5. Conclusions…………………………………………………………………….. 188

5.1 Summary and Discussions………………………………………………….. 188
5.2 Implications…………………………………………………………………. 199
5.3 Limitations………………………………………………………………….. 202
5.4 Recommendations for Future Research…………………………………….. 204

References……………………………………………………………………………. 206

Appendices
A. Instructions and Sample Items of the English Language Test………………… 216
B. Directions and Sample Items of the Test Anxiety Inventory…………………... 218
C. The Test-taking Behavior Survey……………………………………………… 221
D. Permission Letter from School………………………………………………… 224
E. Recruitment Letter to Parents/Guardians……………………………………… 226
F. Recruitment Letter to Students………………………………………………… 228
G. Consent Form for Participation………………………………………………… 230
H. Lower Triangle of the ELT-Item Tetrachoric Correlation Matrix for the
Untimed data……………………………………………………….………. 232
I. Item Difficulty Estimates Based on the Untimed Data………………………… 239

ix
LIST OF TABLES

Table Page
2.1 Aberrant Response Patterns on a Fictitious 10-Item Test………………... 38
3.1 Illustrative Aberrant Response Patterns Associated with Test Anxiety….. 58
3.2 Diagnostic Ability of B* and W* Indices……………………………….. 65

Diagnostic Ability of Klauer’s X2 and X2 …………………………...
3.3 dif ord 70
3.4 Sensitivity of Indices to 4 Types of Aberrant Response Behavior ………. 74
3.5 Critical Values of Selected 7 Indices at 5% and 10% Type I Error Rates.. 78
3.6 Percentage of Aberrant Simulees Classified as Aberrant and Percentage
of Normal Simulees Classified as Aberrant at the .05 and the .10 Type I
Error Rates………………………………………………………………. 79
3.7 Design of Study 1………………………………………………………… 94
3.8 Critical Values of Selected 7 Aberrance Indices at the .10 Alpha Level… 104
4.1 Reliability Indices for Three Instruments………………………………… 116
4.2 Factor Loadings of a One-Factor Solution for 80 ELT Items
(Untimed Data) …………………………………………………………... 120
4.3 Descriptive Statistics for 7 Aberrance Indices …………………………… 122
4.4 Descriptive Statistics for Aberrance Indices by Test Anxiety Level...…… 124
4.5 Descriptive Statistics for Aberrance Indices by Time Group…………….. 125
4.6 Descriptive Statistics for Aberrance Indices by Ability Level…………… 126
x