Complexity, Accuracy, and Fluency as Properties of Language Performance

222 Pages·2012·4.4 MB·English

Complexity, Accuracy, and Fluency as Properties of Language Performance:
The Development of the Multiple Subsystems over Time and in Relation to Each Other

by
Mary Lou Vercellotti
B. A., Carlow University, 1994
M. A., University of Pittsburgh, 2007

Submitted to the Graduate Faculty of
Kenneth P. Dietrich School of Arts and Sciences
in partial fulfillment of the requirements for the degree of
Doctor of Philosophy

University of Pittsburgh
2012
UNIVERSITY OF PITTSBURGH
Kenneth P. Dietrich School of Arts and Sciences

This dissertation was presented

by

Mary Lou Vercellotti

It was defended on
March 16, 2012
and approved by
Dr. Dawn E. McCormick, Lecturer, Linguistics
Dr. Alan Juffs, Associate Professor, Linguistics
Dr. Kevin Hyunkyung Kim, Associate Professor, Psychology in Education
Dr. Nel de Jong, Assistant Professor, Linguistics, VU University Amsterdam
Dissertation Advisor: Dr. Yasuhiro Shirai, Professor, Linguistics

ii
Copyright © by Mary Lou Vercellotti
2012
iii
Complexity, Accuracy, and Fluency as Properties of Language Performance:
The Development of the Multiple Subsystems over Time and in Relation to Each Other

Mary Lou Vercellotti, PhD
University of Pittsburgh, 2012

Applied linguists have identified three components of second language (L2) performance:
complexity, accuracy, and fluency (CAF) to measure L2 development. Many studies researching
CAF found trade-off effects (in which a higher performance in one component corresponds to
lower performance in another) during tasks, often in online oral language performance. Trade-off
effects are attributed to the inability of the learner to simuletaneously attend to all CAF
components at the highest level possible. Although cross-sectional research has suggested that
students at different proficiency levels sacrifice performance in one CAF area while improving
in another, there has been little longitudinal research about CAF (Ortega & Iberri-Shea, 2005).
As such, previous research could not address if CAF grows linearly over time nor if the rate of
CAF growth is the same for all learners. The current study explicitly addresses how language
performance in CAF changes over L2 development in an instructed environment.
This longitudinal study analyzed English L2 oral data from sixty-six students from
Arabic, Chinese, and Korean language backgrounds over 3-9 months in the English Language
Institute at the University of Pittsburgh. Elicited speeches were transcribed, coded, and assessed
with three measures of structural complexity, a measure of lexical variety, two measures of
accuracy, and three measures of fluency. The scores were analyzed with hierarchical linear
modeling (Singer & Willett, 2003) to investigate how each student’s performance changed over
time for each measure and to determine predictive variables. Although individual differences
were found in initial scores (often proficiency differences, but not for all measures), growth
iv
trajectories were the same for all measures, except one grammatical complexity measure (length
of AS unit) where slopes differed by gender. All measures showed growth, and only two
measures (lexical variety and mean length of fluent run) showed non-linear growth. Trade-off
effects found in cross-sectional studies were not found in these longitudinal data even though
within-individual and between-individual correlations were also calculated. Additionally, the
results may suggest that instructed language performance growth is uniform, rather than along
individual paths. The research also serves to evaluate the measures, which has research and
pedagogical implications.

v
TABLE OF CONTENTS
TABLE OF CONTENTS ........................................................................................................... VI
LIST OF TABLES ................................................................................................................... XIII
LIST OF FIGURES ................................................................................................................. XVI
PREFACE ............................................................................................................................... XVII
1.0 INTRODUCTION............................................................................................................. 1
2.0 MEASURING COMPLEXITY, ACCURACY, AND FLUENCY ............................... 4
2.1 BASIC UNIT IN SPOKEN DATA ..................................................................... 4
2.1.1 The (Undefined) Utterance ........................................................................... 5
2.1.2 Defining Utterances by Syntactic Criteria .................................................. 7
2.1.2.1 Analysis of Speech Unit ........................................................................ 8
2.2 ACCURACY ........................................................................................................ 9
2.2.1 Measuring Accuracy ..................................................................................... 9
2.2.1.1 Specific Measures ................................................................................ 10
2.2.1.2 General Measures of Accuracy .......................................................... 11
2.3 COMPLEXITY .................................................................................................. 14
2.3.1 Complexity Measures ................................................................................. 14
2.3.1.1 Complexity by Sophistication ............................................................ 15
2.3.1.2 Grammatical Complexity ................................................................... 16
vi
2.3.1.3 Lexical Variety Measures ................................................................... 18
2.3.2 Summary of Complexity Measures ........................................................... 20
2.4 FLUENCY .......................................................................................................... 21
2.4.1 Fluency Measures........................................................................................ 21
2.4.1.1 Fluency Breakdown - Pausing ........................................................... 22
2.4.1.2 Fluency Proceduralization - Mean Length of Fluent Run .............. 23
2.5 SUMMARY OF MEASURING CAF .............................................................. 24
3.0 RELATIONSHIPS BETWEEN COMPLEXITY, ACCURACY, AND FLUENCY 26
3.1 TRADE-OFF EFFECTS IN LANGUAGE PERFORMANCE ..................... 27
3.1.1 Trade-off Effects Predicted in Language Performance .......................... 29
3.1.2 Connected Growers .................................................................................... 32
3.1.3 Context of the Previous Findings............................................................... 34
3.1.3.1 Task Instructions................................................................................. 34
3.1.3.2 Effect of Task and Research Design .................................................. 35
3.1.4 Summary of Trade-off Effects ................................................................... 36
3.2 LANGUAGE PERFORMANCE OVER TIME ............................................. 38
3.2.1 Growth within CAF Constructs over Time .............................................. 38
3.2.2 Growth across CAF Constructs over Time .............................................. 42
3.3 INDIVIDUAL DIFFERENCES IN LANGUAGE PERFORMANCE AND
GROWTH ........................................................................................................................... 44
3.3.1 Affective factors .......................................................................................... 45
3.3.2 Age ................................................................................................................ 45
3.3.3 Gender .......................................................................................................... 46
vii
3.3.4 Initial Proficiency ........................................................................................ 46
3.3.5 Language Background................................................................................ 47
3.3.6 Learner Orientation.................................................................................... 48
3.4 SUMMARY OF LANGUAGE PERFORMANCE ......................................... 50
3.5 SUMMARY OF THE ISSUES ......................................................................... 51
3.5.1 Trade-off Effects ......................................................................................... 52
3.5.2 Language Development .............................................................................. 52
3.5.3 Remaining Issues ......................................................................................... 53
3.5.3.1 Research Questions ............................................................................. 54
4.0 THE STUDY.................................................................................................................... 57
4.1 METHODOLOGY ............................................................................................ 57
4.2 PARTICIPANTS ............................................................................................... 57
4.2.1 Instruction Cohort 1 ................................................................................... 59
4.2.2 Instruction Cohort 2 ................................................................................... 60
4.2.3 Comparison and Summary of Cohorts ..................................................... 60
4.2.4 Comparison of Language Background Groups ....................................... 61
4.2.5 Language Background................................................................................ 62
4.2.5.1 Arabic Language Background English Learners ............................ 63
4.2.5.2 Chinese Language Background English Learners ........................... 64
4.2.5.3 Korean Language Background English Learners ........................... 65
4.3 THE ENGLISH LANGUAGE INSTITUTE ................................................... 66
4.3.1 Placement Tests ........................................................................................... 67
4.4 DATA .................................................................................................................. 68
viii
4.4.1 Data Collection ............................................................................................ 68
4.4.2 Data Transcription and Coding ................................................................. 71
4.4.3 Dependent Variables ................................................................................... 72
4.4.3.1 Accuracy Measures ............................................................................. 73
4.4.3.2 Complexity Measures ......................................................................... 74
4.4.3.3 Fluency Measures ................................................................................ 74
4.5 DATA ANALYSIS ............................................................................................. 75
4.5.1 Quantifiable Variables Study Design ........................................................ 75
4.5.2 Hierarchical Linear Modeling ................................................................... 77
4.5.2.1 Nonparametric .................................................................................... 77
4.5.2.2 Parametric ........................................................................................... 78
4.5.2.3 Conditioned (with Predictors) Models .............................................. 79
4.5.2.4 Rationale for Using Hierarchical Linear Modeling ......................... 81
4.5.3 Correlations Analysis.................................................................................. 82
4.5.3.1 Between-individual Correlations ....................................................... 83
4.5.3.2 Within-individual Correlations ......................................................... 84
4.5.3.3 Interpretation of Correlations ........................................................... 84
4.6 SUMMARY OF METHODOLOGY ............................................................... 85
5.0 RESULTS ........................................................................................................................ 87
5.1 ACCURACY ...................................................................................................... 88
5.1.1 Percentage of Error-free AS units (A1) .................................................... 88
5.1.2 Percentage of Error-free Clauses (A2) ...................................................... 94
5.1.3 Correlations between the Accuracy Measures ......................................... 98
ix
5.1.4 Discussion of Accuracy Results.................................................................. 98
5.2 COMPLEXITY ................................................................................................ 101
5.2.1 Length of AS unit (C1) ............................................................................. 102
5.2.2 Clause Length (C2) ................................................................................... 105
5.2.3 Clauses per AS unit (C3) .......................................................................... 108
5.2.4 Lexical Variety (C4) .................................................................................. 110
5.2.5 Correlations among Complexity Measures ............................................ 113
5.2.6 Discussion of Complexity Results ............................................................ 115
5.2.6.1 Grammatical Complexity ................................................................. 115
5.2.6.2 Lexical Variety .................................................................................. 118
5.2.6.3 Correlations within Complexity ...................................................... 121
5.3 FLUENCY ........................................................................................................ 123
5.3.1 Phonation Time Ratio (F1) ....................................................................... 124
5.3.2 Mean Length of Pause (F2) ...................................................................... 128
5.3.3 Mean Length of Fluent Run (F3) ............................................................. 132
5.3.4 Correlations among Fluency Measures .................................................. 135
5.3.5 Discussion of Fluency Results .................................................................. 137
5.3.5.1 Predictors of Fluency ........................................................................ 137
5.3.5.2 Unexplained Variance....................................................................... 138
5.3.5.3 Growth Trajectories ......................................................................... 139
5.3.5.4 Correlations within Fluency ............................................................. 140
5.4 CORRELATIONS BETWEEN CAF CONSTRUCTS ................................ 140
5.4.1 Accuracy and Complexity ........................................................................ 141
x