Effects of age and schooling on 22 ability and achievement tests

144 Pages·2017·2.66 MB·English

University of Iowa
Iowa Research Online
Theses and Dissertations
Spring 2013
Effects of age and schooling on 22 ability and
achievement tests
James Lamar Gambrell
University of Iowa
Copyright 2013 James Lamar Gambrell
This dissertation is available at Iowa Research Online: https://ir.uiowa.edu/etd/2498
Recommended Citation
Gambrell, James Lamar. "Effects of age and schooling on 22 ability and achievement tests." PhD (Doctor of Philosophy) thesis,
University of Iowa, 2013.
https://doi.org/10.17077/etd.om72sh4k
Follow this and additional works at:https://ir.uiowa.edu/etd
Part of theEducational Psychology Commons
EFFECTS OF AGE AND SCHOOLING ON 22 ABILITY AND ACHIEVEMENT
TESTS
by
James Lamar Gambrell
An Abstract
Of a thesis submitted in partial fulfillment
of the requirements for the Doctor of
Philosophy degree in Psychological and Quantitative Foundations
in the Graduate College of
The University of Iowa
1

May 2013
Thesis Supervisor: Professor David F. Lohman
1

ABSTRACT
Although much educational research has investigated the relative effectiveness of
different educational interventions and policies, little is known about the absolute net
benefits of K-12 schooling independent of growth due to chronological age and out-of-
school experience. The nearly universal policy of age tracking in schools makes this
topic difficult to investigate. However, a quasi-experimental regression discontinuity
design can be used to separate observed test score differences between grades into
independent age and schooling components, yielding an estimate of the net effects of
school exposure at each grade level. In this study, a multilevel version of this design was
applied to scores on 22 common ability and achievement tests from two major
standardized test batteries. The ability battery contained 9 measures of verbal,
quantitative, and figural reasoning. The achievement battery contained 13 measures in
the subjects of language arts, mathematics, reading, social studies, science, and use of
reference materials. The analysis was based on sample of over 20,000 students selected
from a longitudinal database collected by a large U.S. parochial school system.
The theory of fluid (Gf) and crystallized (Gc) intelligence predicts that these tests
will show systematically different levels of sensitivity to schooling. Indeed, the
achievement (Gc) tests were found to be nearly three times more sensitive to schooling
than they were to aging (one-year effect sizes of .41 versus .15), whereas the ability (Gf)
1
tests were equally influenced by age (.19) and schooling (.20). Nonetheless, the
schooling effect on most Gf tests was substantial, especially when the compounding over
a typical school career is considered. This replicates the results of previous investigations
of age and schooling using regression discontinuity methods and once again contradicts
common interpretations of fluid ability as a fixed quantity.
Different measures within a domain exhibited wide variation in their degree of
school sensitivity. Those tests that were less sensitive to schooling generally required
reading, reasoning, transfer, synthesis, or translation between symbol systems; posed a
2

wider range of questions; and/or presented problems in an unfamiliar format.
Quantitative reasoning tests showed more sensitivity to schooling than figural reasoning
tests, while verbal reasoning tests occupied a middle ground between the two. Schooling
had the most impact on basic arithmetic skills and knowledge of mathematical concepts,
and had much weaker impact on the solution of math word problems. School-related
gains on isolated language skills were much larger than gains on solving grammar and
style problems in context. The weakest schooling impact overall was on reading
comprehension where effects were no larger than those for fluid verbal ability.
An interesting dichotomy was found between spelling and paper folding (a
measure of figural and spatial reasoning). Spelling skills showed robust schooling effects
but a consistently negative age slope, a puzzling result which indicates that younger
students in each group outperformed older students. Paper folding showed the opposite
pattern, a large age effect and a small but consistently negative schooling effect.
Results serve to rebut skepticism about both the impact of schooling on test scores
and the validity of distinctions between ability and achievement measures. It is argued
that the regression discontinuity design has great potential in the measurement of school
effectiveness, while also offering a source of validity evidence for test developers and test
users. Implications for theories of cognitive ability and future research on schooling 2

effects are discussed.

Abstract Approved: ____________________________________
Thesis Supervisor

____________________________________
Title and Department

____________________________________
Date
EFFECTS OF AGE AND SCHOOLING ON 22 ABILITY AND ACHIEVEMENT
TESTS
by
James Lamar Gambrell
A thesis submitted in partial fulfillment
of the requirements for the Doctor of
Philosophy degree in Psychological and Quantitative Foundations
in the Graduate College of
The University of Iowa
1
May 2013
Thesis Supervisor: Professor David F. Lohman
Copyright by
JAMES LAMAR GAMBRELL
2013
All Rights Reserved
2
Graduate College
The University of Iowa
Iowa City, Iowa
CERTIFICATE OF APPROVAL
_______________________
PH.D. THESIS
_______________
This is to certify that the Ph.D. thesis of
James Lamar Gambrell
has been approved by the Examining Committee
for the thesis requirement for the Doctor of Philosophy
degree in Psychological and Quantitative Foundations at the May 2013
graduation.
Thesis Committee: ___________________________________
David F. Lohman, Thesis Supervisor
___________________________________
Michael J. Kolen
___________________________________
Ernest T. Pascarella
___________________________________
Donald B. Yarbrough
___________________________________
Mary K. Cowles
To Amanda
2

ii
ACKNOWLEDGMENTS
I would like to thank Dave Lohman for all his support these past six years; it has
been a wonderful experience working with you here at Iowa. Thank you E. and J. for
collecting this amazing dataset and allowing me to use it. Thanks also to Ryan for his
encouragement and advice early on in this process, and to my committee members for
their kind comments. I am also grateful to Allen Doolittle, Joseph Betts, and others at
Riverside Publishing for their support and interest in my work.
3

iii
ABSTRACT
Although much educational research has investigated the relative effectiveness of
different educational interventions and policies, little is known about the absolute net
benefits of K-12 schooling independent of growth due to chronological age and out-of-
school experience. The nearly universal policy of age tracking in schools makes this
topic difficult to investigate. However, a quasi-experimental regression discontinuity
design can be used to separate observed test score differences between grades into
independent age and schooling components, yielding an estimate of the net effects of
school exposure at each grade level. In this study, a multilevel version of this design was
applied to scores on 22 common ability and achievement tests from two major
standardized test batteries. The ability battery contained 9 measures of verbal,
quantitative, and figural reasoning. The achievement battery contained 13 measures in
the subjects of language arts, mathematics, reading, social studies, science, and use of
reference materials. The analysis was based on sample of over 20,000 students selected
from a longitudinal database collected by a large U.S. parochial school system.
The theory of fluid (Gf) and crystallized (Gc) intelligence predicts that these tests
will show systematically different levels of sensitivity to schooling. Indeed, the
achievement (Gc) tests were found to be nearly three times more sensitive to schooling
than they were to aging (one-year effect sizes of .41 versus .15), whereas the ability (Gf)
4
tests were equally influenced by age (.19) and schooling (.20). Nonetheless, the
schooling effect on most Gf tests was substantial, especially when the compounding over
a typical school career is considered. This replicates the results of previous investigations
of age and schooling using regression discontinuity methods and once again contradicts
common interpretations of fluid ability as a fixed quantity.
Different measures within a domain exhibited wide variation in their degree of
school sensitivity. Those tests that were less sensitive to schooling generally required
reading, reasoning, transfer, synthesis, or translation between symbol systems; posed a
iv