ERIC EJ773367: A Study of the Association of Autonomy and Achievement on Performance

2007·0.08 MB·English

by  ERIC

Robert H. Tai, Philip M. Sadler, Adam V. Maltese
A Study of the Association of Autonomy
and Achievement on Performance
The authors fi nd that autonomous learning activities in high school science
interact with high school mathematics grades to produce a signifi cant
association with college science grades.
Inquiry-based instructional prac- types of studies are not common (e.g. much autonomy they reported having
tices are a mainstay of the National Cronbach & Snow, 1977). With more in high school science through labs and
Science Education Standards (Na- students going to college (Bureau of projects. Our objective was to see if
tional Research Council, 1996). The Labor Statistics, 2005) than during the students who reported experiencing
National Research Council (NRC) past few years, and most high school more or less self-directed projects and
teachers’ guide asks the critical ques- science teachers naturally empha- labs performed differently in college
tion, “How does a teacher decide sizing college preparation (Hoffer, science when we took into account
how much guidance to provide in an Quinn, & Suter, 1996), one option for a their prior academic background. To
inquiry?” (NRC, 2000, p. 30). Another long-range measure of performance is provide a more solid foundation for our
primary concern is the quality of stu- introductory college science. Research conclusions, we performed the same
dent work produced in these activities. linking high school preparation to col- analysis on three different data sets in
For many teachers who assign inquiry lege performance may provide some biology, chemistry, and physics.
activities, the reality is that while some insight into best practice.
Methodology
students may produce good work, This study investigated the inter-
others languish (O’Neill & Polman, action between students’ academic The data used in this study is a sub-
2004; Polman, 2000). background (high school grades, stan- sample taken from a national survey
A major fi nding of prior interaction dardized exams, and enrollment in ad- entitled Factors Infl uencing College
research was that higher achievers vanced high school courses) and how Science Success (Project FICSS,
responded better in less-structured NSF-REC 0115649). A sample of 67
learning environments, such as stu- four-year colleges and universities was
dent designed projects and labs, while A major fi nding of prior selected from a comprehensive list of
lower achievers responded better to interaction research was nearly 1,700, using stratifi ed random
more-structured environments, as in sampling based on size to insure that
that higher achievers
labs using worksheets and detailed the sample spanned the range from
responded better in
directions (Cronbach & Snow, 1977; small colleges to large universities.
less-structured learning
Tobias, 1981). Based on this fi nding, Of the selected schools, 55 schools
optimal levels of academic perfor- environments, such as from 31 states participated.
mance would be expected if instruc- student designed projects Faculty in 29 biology departments,
tional methods were chosen to more and labs, while lower 31 chemistry departments, and 37
closely match students’ backgrounds. achievers responded physics departments participated
Will matching a student’s academic and data was collected from college
better to more-structured
achievement with particular teaching science students in 128 different fi rst
environments, as in labs
practices have a long-range impact on semester introductory college science
using worksheets and
their academic performance? A review courses all taught exclusively in the
of existing literature shows that these detailed directions. Fall Semesters of 2002 and 2003. Insti-
22 SCIENCE EDUCATOR
Table 1: Summary of Institutional Characteristics of Participating Schools impute missing data for the predictors:
SAT-Mathematics, SAT-Verbal, Last
Field (# of Participant Affi liation Avg. ACT Avg. SAT School Size a # of
High School Mathematics Grade, Last
Schools) States
Range of Range of High School Science Grade, and Last
Totals Public Private Avg. ACT Avg. SAT S M L High School English Grade (Allison,
Physics (37) 1903 24 13 17-30 830-1320 18 12 7 26 2002; Little & Rubin, 2002; Scheffer,
Chemistry (31) 3521 20 11 17-27 830-1210 16 11 4 22
2002). With data imputation, 88%
Biology (29) 2749 19 10 17-30 840-1320 13 9 8 23
of the surveys were retained for the
a Small schools < 5,000 student enrollment (full-time equivalent student enrollment analysis. The fi nal sample sizes were
totals, FTE), medium-size schools between 5,000 and 15,000 FTE, and large schools
2,430 students for biology; 3,187 stu-
>15,000 FTE.
dents for chemistry, and 1,577 students
tutional data is displayed in Table 1. To college students are reasonably accu-
for physics.
check for institutional “self-selection” rate and produce valid information. To
The fi rst step in data analysis was
bias, we compared participating and improve accuracy and reliability, the
a descriptive comparison across dif-
non-participating schools across mea- survey was designed with character-
ferent demographic and general edu-
sures such as school size, admissions istics associated with improving recall
cational background variables. Next,
selectivity, and geographic location (Niemi & Smith, 2003; Sawyer, Laing
multiple linear regression models were
and found no indications of bias. & Houston, 1988; Schiel & Noble,
fi tted to the outcome variable, fi nal col-
For continuity in comparison across 1991; Valiga, 1987). The reliability
lege science course grades, hereafter
courses, we chose to include only of the questionnaire was assessed
referred to as GRADES. Controlling
courses with the ubiquitous large through a separate test-retest study
for differences in demographic and
lecture format, by far the most likely of 113 introductory college chemistry
general educational backgrounds, this
to be experienced by high school stu- students, not included in the sample
analysis included students’ academic
dents who take introductory college analyzed here. The reliability study
background measures (high school
science. All courses in this survey required students to complete the
grades, standardized exams, and pat-
fi lled program requirements for majors survey on two separate occasions, two
terns of advanced course taking), and
within their respective disciplines. All weeks apart. The resulting reliability
students’ experiences with inquiry-
67 schools originally selected for the coeffi cients ranged from 0.46 to 0.69,
related pedagogies (Student-designed
survey used this class format. The which were considered reasonably
Projects and Level of Freedom in
total sample sizes were: 2,754 biology high for analyses of groups of 100
laboratory exercises).
surveys, 3,521 chemistry surveys, and students (Thorndike, 1997). Finally,
1,903 physics surveys. These sample to further enhance accuracy, the sur-
sizes offer a high degree of statisti- veys were administered during college
cal power (Light, Singer, & Willett, science class sessions, (i.e. lectures, Research linking high
1990). recitation meetings, or lab sessions) school preparation to
The survey instrument was de- and the students’ fi nal grades were
college performance may
signed by the researchers to collect reported by professors.
provide some insight into
information about a large number of As is common in many surveys,
best practice.
curricular issues in high school sci- not every participant answered every
ence. The survey was vetted through question. Many students left blanks, or
a series of focus group interviews marked multiple responses to the same College science grades are a com-
and pilot surveys. For retrospective question. Recommended research mon choice to gauge science perfor-
self-report surveys, limitations of ac- practice favors data imputation over mance (Gainen & Willemsen, 1995;
curacy and reliability are important the more commonly used tactic of Ozsogomonoyan & Loftus, 1979;
considerations. A review of research list-wise deletion (Peugh & Enders, Spencer, 1996). A review of course
(Kuncel, Credé, & Thomas, 2005) 2004). We employed the Expectation syllabi shows that college grades are
concluded that self-report surveys of – Maximization (EM) Algorithm1 to not a single measure, but a composite
1 An extended explanation of the process of data imputation is available upon request from the corresponding author.
SPRING 2007 VOL. 16, NO. 1 23
The demographic background pre- disciplines at about one per high school
dictors included gender, racial/ethnic course (see Table 2). However, Level
Structured learning
background, parental education levels, of Freedom in Designing/Conducting
activities are essential to
average county household income, Labs varied widely between students
building the knowledge-
and high school type (i.e. public in biology and the two physical sci-
base necessary for or private) which past studies have ences. The biology ratings appear to
understanding more shown to be important (e.g. Bryk, be nearly half of a standard deviation
advanced scientifi c Lee, & Holland, 1993; Burkam, Lee, below the other averages. This result
& Smerdon, 1997). suggests that college chemistry and
concepts, while autonom-
physics students reported more free-
ous learning forms the
Results and Discussion
dom in their corresponding high school
foundation of scientifi c
The descriptive statistics for the classes than college biology students
inquiry. predictor variable, Number of Own reported about their high school biol-
Projects, were very similar across all ogy classes.
three data sets, with the average num- The mathematics achievement
of several different measures (e.g. ber reported by students in all three measures show some differences
tests, quizzes, homework sets, and
exams) collected over months, and
Table 2: Descriptive Statistics for Continuous Predictors
as a result, are collectively more in-
Min Max Biology Chemistry Physics
dicative of student performance than
Mean (s.d.) Mean (s.d.) Mean (s.d.)
a single achievement test. In addition, Number of Student-des igned Projects
college-level content subsumes high (None = 0, More than 3 = 4) 0 4 1.2 (1.2) 1.0 (1.2) 1.2 (1.3)
Level of Laboratory Freedom
school content, often in just a few
(None = 0, Complete = 4) 0 4 1.2 (1.2) 1.6 (1.3) 1.7 (1.2)
weeks. Therefore, one would expect
SAT Score Quantitative 220 790 580 (100) 590 (100) 620 (90)
that students with a deeper conceptual Verbal 200 800 570 (100) 570 (100) 580 (100)
Last HS Grade in …
understanding of the fundamental sci-
(A = 5, F = 1) Mathematics 2 5 4.3 (0.8) 4.3 (0.8) 4.5 (0.7)
ence topics would have an advantage Science 1 5 4.4 (0.8) 4.4 (0.8) 4.5 (0.7)
and earn higher college grades. To English 1 5 4.6 (0.6) 4.6 (0.6) 4.6 (0.6)
address concerns about comparability Highest Parent Education Level 0 4 2.8 (1.1) 2.7 (1.1) 2.9 (1.0)
(Attended HS = 0, Grad. School = 4)
across courses and institutions, col-
lege effects variables were included
to account for these differences (Pike Table 3: Frequency Statistics for Categorical Predictors
& Saupe, 2002).
Predictors Number of Students
The academic achievement pre- Biology Chemistry Physics
dictors included SAT-Quantitative AP® Science Did Not Enroll 2428 88% 3155 90% 1747 92%
Enrolled 326 12% 366 10% 156 8%
and Verbal scores; Last High School
HS Calculus No HS Calculus 1775 65% 2000 57% 842 44%
grade in Mathematics, Science, and
Regular 373 14% 518 15% 351 18%
English; high school enrollment in AP® A/B 473 17% 792 22% 526 28%
calculus (regular, Advanced Place- AP® B /C 133 5% 211 6% 184 10%
ment® Calculus A/B, and Advanced Race/Ethnicity Native American 33 1% 36 1% 27 1%
Black 207 8% 202 6% 102 5%
Placement® Calculus B/C); and
Hispanic 134 5% 177 5% 89 5%
enrollment in Advanced Placement® Asian 167 6% 293 8% 151 8%
Multi-racial 73 3% 93 3% 38 2%
science courses. The inquiry-type
Not pRoerted 53 2% 95 3% 106 6%
learning activity predictors included White 2087 76% 2625 75% 1390 73%
Number of Student-designed Projects Year n i oCllege Freshman 1496 54% 2103 60% 234 12%
and Level of Freedom in Designing/ Sophomore 691 25% 811 23% 733 39%
Junior 404 15% 393 11% 579 30%
Conducting Labs.
Senior 127 5% 143 4% 273 14%
No sRpeonse 34 1% 64 2% 82 4%
24 SCIENCE EDUCATOR
across the three samples. As one of the sample in each discipline (see biology. This artifact suggests that
might expect, physics students report Table 3). For Year in College, biology fewer students with low and very
having taken more mathematics and and chemistry students were primarily low Last High School Mathematics
also report generally higher math- freshman, while physics student were Grades enrolled in college physics.
ematics achievement. The percent- primarily sophomores, which may be Therefore, it seems reasonable to argue
age of students reporting high school due to the calculus co-/prerequisite that the interaction was not found in
calculus enrollment varied across the for some physics courses. In general, the physics data because students with
three disciplines: 35.4 % for biology, the descriptive data is consistent with weaker mathematics achievement in
43.2 % for chemistry, and 55.8 % for well-known educational trends. This high school may choose not to enroll
physics students. The average SAT- consistency is important, since our in physics courses.
Mathematics scores (see Table 2) for analysis is intended to produce gen- At fi rst glance, the graphs in Fig-
biology and chemistry students were eralizable fi ndings. ure 1 appear to indicate that students
580 and 590, respectively; while the who reported complete freedom in
Regression Models
average SAT-Mathematics score for designing and conducting labs were
The main focus of this study was
physics students was 620. Also, biol- at a disadvantage in all disciplines.
to investigate the interaction between
ogy and chemistry students’ average However, a closer look shows that
differences in students’ academic
high school mathematics grades were large differences are mainly for stu-
achievement and their high school
the same at 4.3 (A = 5; B = 4), while dents reporting low to very low Last
learning experiences. We wanted to
the average for physics students was High School Mathematics Grades in
study what infl uence the level of au-
4.5, roughly one quarter of a standard the biology and chemistry. For the
tonomy students reported experienc-
deviation higher. In general, introduc- students in the high and moderate
ing in high school had on their college
tory college physics students were groups, the differences are much
science performance when taking into
higher math achievers than introduc- more modest with results similar to
account their academic background,
tory college biology and chemistry the physics analysis which showed
specifi cally with respect to mathemat-
students. no interaction. The regression results
ics achievement. The analysis found
predict that students earning low and
a signifi cant interaction between Last
very low Last High School Mathemat-
High School Mathematics Grade and
Autonomous learning is the ics Grades who also reported Complete
Level of Lab Freedom for the biology
seed of scientifi c research. Lab Freedom were predicted to have
(α – level = 0.05) and chemistry (α
college biology and chemistry grades
– level = 0.05) analyses, but not for
approximately one half of a letter grade
In other measures, we found very the physics analysis. Figure 1 presents
lower than their peers who reported
similar results across the three separate a comparison of the results of the
no freedom. These results imply that
samples. A comparison of the averages multiple linear regression analysis.
higher autonomy in high school labs
for SAT-Verbal, Last HS Grades in Sci- All three regression models accounted
may not be the best way for students
ence and English, and Highest Parental for roughly one third of the overall
with low mathematics grades to learn
Educational Level found them all to variance, a strong result for such large-
science in high school.
be similar. Fewer than 12% of the stu- scale analyses.2
dents in all disciplines enrolled in the At fi rst glance, these fi ndings may
corresponding Advanced Placement® seem inconsistent with results for Conclusions
science course in high school. A com- physics lacking a signifi cant interac- The results from this study suggest
parison of gender differences showed tion. However, looking back at com- that decisions about how much free-
more females than males in biology parisons of the students’ backgrounds dom to give students in high school sci-
and chemistry, and more males than fe- in Tables 2 and 3, the average college ence activities such as labs and projects
males in physics. Students enrolled in physics student appears to have higher should also include considerations of
introductory sciences are overwhelm- levels of mathematical achievement students’ achievement in mathemat-
ingly white, comprising three quarters than his or her peers in chemistry or ics. Structured learning activities are
2 Tables comparing the three replicate multiple linear regression models are available from the fi rst author upon request.
SPRING 2007 VOL. 16, NO. 1 25
Figure 1: Association of Level of Lab Freedom (None to Complete) and Last HS Mathematics Grade on Predicted Introductory
College Science Grade
85 85
Grade Grade
ology 80 mistry 80
e Bi Che
Predicted Introductory Colleg 75 NLMooowndeerate Predicted Introductory College 75 NLMooowndeerate
High High
Complete Complete
70
70
Very Low Low Moderate High Very Low Low Moderate High
Last HS Mathematics Grade (High = A; Very Low = D or F) Last HS Mathematics Grade (High = A; Very Low = D or F)
90 with lower levels of high school mathematics achievement
had greater success in college science when they reported
more structured high school lab experiences. Students with
e higher levels of high school mathematics achievement did
d
a
Gr not reveal much variation with differences in lab structure.
s
sic 85 These results agree with earlier research and extend these
hy
e P conclusions to longer range outcomes.
g
olle This study compelled us to think more deeply about
C
ory teaching and learning science, and the outcomes. Final
duct course grades were chosen for this analysis primarily
o
d Intr 80 None because of their clear impact on students’ career paths;
e
dict Low however, other outcomes are also important to consider.
e
Pr Moderate While some forms of teaching may be highly effective in
High building students’ background knowledge and enhancing
Complete their performance in science, other methodologies seem to
75 raise students’ interest in learning science and spark their
Very Low Low Moderate High
imagination, possibly contributing to their continuance in
Last HS Mathematics Grade (High = A; Very Low = D or F)
the study of science.3 Performance and continuance may be
essential to building the knowledge-base necessary for two different dimensions of science pedagogy. Other stud-
understanding more advanced scientifi c concepts, while ies have found important positive impacts of instructional
autonomous learning forms the foundation of scientifi c methods on student interest and attitudes (e.g. Hofstein,
inquiry. The fi ndings from this study suggest that students Shore, & Kipnis, 2004; O’Neill & Polman, 2004).
3 The term “continuance” is used here rather than the term “persistence,” which refers to long-term student commitment in the pursuit
of an educational goal (e.g. Seymour & Hewitt, 1997). This study captures only shorter-term course-taking. Science course-tak-
ing is but one step in science persistence. We wish to acknowledge Mary M. Atwater for her insight (Personal correspondence,
May 16, 2005).
26 SCIENCE EDUCATOR
Burkam, D. T., Lee, V. E., & Smerdon, B. National Research Council. (2000). Inqui-
A. (1997). Gender and science learning ry and the National Science Education
This study compelled us to
early in high school: Subject matter Standards: A guide for teaching and
think more deeply about and laboratory experiences. American learning. Washington, DC: National
teaching and learning Educational Research Journal. 34(2), Academy Press.
297 – 331. Niemi, R. G., & Smith, J. (2003). The
science, and the outcomes.
Cronbach, L. J., & Snow, R. E. (1977). accuracy of students’ reports of course
Aptitudes and instructional meth- taking in the 1994 National Assessment
If freedom in laboratory design is ods: A Handbook for Research on of Educational Progress. Educational
associated with lower college science Interactions. New York: Irvington Measurement: Issues and Practice.
performance for many students, should Publishers. 22(1), 15 – 21.
Gainen, J., & Willemsen, E. W. (Eds.). O’Neill, D. K., & Polman, J. L. (2004).
this approach to instruction be aban-
(1995). Fostering student success in Why educate “little scientists?”
doned in high school science courses?
quantitative gateway courses. No. 61. Examining the potential of practice-
Abandoning freedom for students to
San Francisco: Jossey-Bass. based scientifi c literacy. Journal of
design and conduct their own experi-
Hodson, D. (1996). Laboratory work Research in Science Teaching. 41(3),
ments in science labs is very short-
as scientifi c method: Three decades 234 – 266.
sighted. Autonomous learning is the of confusion and distortion. Journal Ozsogomonoyan, A., & Loftus, D. (1979).
seed of scientifi c research. Scientists of Curriculum Studies. 28(2), 115 Predictors of general chemistry grades.
must achieve some level of research – 135. Journal of Chemical Education. 55(3),
independence in order to make con- Hoffer, T. B., Quinn, P., & Suter, L. E. 173 – 175.
tributions to the knowledge-base. (1996). High School Seniors’ Instruc- Pike, G. R., & Saupe, J. L. (2002). Does
tional Experiences in Science and high school matter? An analysis of
Certainly, there have been questions
Mathematics: National Educational three methods of predicting fi rst-year
raised about the authenticity of stu-
Longitudinal Study of 1988. (NCES grades. Research in Higher Education.
dents’ laboratory work as a refl ection
95-278). Washington, DC: Government 43(2), 187 – 207.
of scientifi c practice (Hodson, 1996).
Printing Offi ce. Polman, J. L. (2000). Designing project-
However, laboratory work in school
Hofstein, A., Shore, R., & Kipnis, M. based science: Connecting learners
science is a pedagogical tool to teach (2004). Providing high school chem- through guided inquiry. New York:
both content and practice in scientifi c istry students with opportunities to de- Teachers College Press.
inquiry, thus a balance must be struck velop learning skills in an inquiry-type Peugh, J. L., & Enders, C. K. (2004).
between structure and autonomy in laboratory: A case study. International Missing data in educational research:
inquiry-type learning activities. Journal of Science Education. 26(1), A review of reporting practices and
47 – 62. suggestions for improvement. Review
References Kuncel, N. R., Credé, M., & Thomas, L. of Educational Research, 74(4), 525
L. (2005). The validity of self-reported – 556.
Allison. P. D. (2002). Missing data: grade point averages, class ranks, and Sawyer, R., Laing, J., & Houston, M.
Quantitative applications in the social test scores: A meta-analysis and review (1988). Accuracy of self-reported high
sciences. Thousand Oaks, CA: Sage of the literature. Review of Educational school courses and grades of college-
Publications. Research. 75(1), 63 – 82. bound students. ACT research Report
Bryk, A. S., Lee, V. E., & Holland, P. B. Light, R. J., Singer, J. D., & Willett, J. B. Series 88-1. Iowa City, IA: American
(1993). Catholic schools and the com- (1990). By design: Planning research College Testing Program.
mon good. Cambridge, MA: Harvard on higher education. Cambridge, MA: Scheffer, J. (2002). Dealing with missing
University Press. Harvard University Press. data. Research Letters in the Informa-
Bureau of Labor Statistics. (2005, March). Little, R. J. A., & Rubin, D. B. (2002). tion and Mathematical Sciences, 3,
News release: College enrollment and Statistical analysis with missing data. 153–160.
work activity of 2004 high school New York: Wiley. Schiel, J., & Noble, J. (1991). Accuracy
graduates. (USDL 05-487). Retrieved National Research Council. (1996). Na- of self-reported course work and grade
on July 25, 2005 from http://www.bls. tional science education standards. information of high school sopho-
gov/news.release/hsgec.nr0.htm Washington, DC: National Academy mores. ACT Research Report Series
Press. 91-6. Iowa City, IA: American College
Testing Program.
SPRING 2007 VOL. 16, NO. 1 27
Spencer, H. E. (1996). Mathematical
Robert H. Tai is an assistant professor of sci- Acknowledgements: We are indebted to the
SAT test scores and college chemistry
ence education at the University of Virginia. professors at universities and colleges na-
grades. Journal of Chemical Education.
Correspondence concerning this article may tionwide who felt that this project was worth
73(12), 1150 – 1153.
be sent to [email protected]. giving over a piece of their valuable class to
Thorndike, R. M. (1997). Measurement administer our surveys and their students’
and evaluation in psychology and Philip M. Sadler is the director of the Science willingness to answer our questions. This work
education (6th ed.) Upper Saddle River, Education Department and senior lecturer on has been carried out under a grant form the
NJ: Merrill. celestial navigation at the Harvard-Smithsonian Interagency Educational Research Initiative
Tobias, S. (1981). Adapting instruction to Center for Astrophysics. (NSF-REC 0115649). Any opinions, fi ndings
individual difference among students. and conclusions or recommendations expressed
Educational Psychologist, 61(2), 111 Adam V. Maltese is a doctoral student in science in this material are those of the authors and do
– 120. education at the University of Virginia. not necessarily refl ect the views of the National
Science Foundation, The U.S. Department
Valiga, M. J. (1987). The accuracy of self-
of Education, or the National Institutes of
reported high school course and grade
Health.
information. ACT Research Report
Series 87-1. Iowa City, IA: American
College Testing Program.
28 SCIENCE EDUCATOR