ERIC EJ783418: Advanced Placement Exam Scores as a Predictor of Performance in Introductory College Biology, Chemistry and Physics Courses

2007·1 MB·English

by  ERIC

Philip M. Sadler, Robert H. Tai
Advanced Placement Exam Scores
as a Predictor of Performance in
Introductory College Biology,
Chemistry and Physics Courses
Survey data from 8,594 students in 55 randomly chosen colleges and universi-
ties finds that those having passed an AP science exam earn somewhat higher
college science grades, but not enough to assume prior mastery. Moreover,
half of this performance difference appears to be related to demographics
and high school coursework and not to students’ AP coursework.
Introduction grades of 3 are equivalent to the outcome of AP course-taking itself
Students who take Advanced a range of work representing may be overestimated. Dougherty,
Placement (AP) examinations in high mid-level B to mid-level C Mellor and Jian (2006a) offered the
school seek the option of receiving performance in college (College following insight:
credit for and bypassing introductory Board, 2006, p. ). Much of those [AP] students
college level courses. These exams are The purpose of this study is to ap- later success in college may
constructed and graded by the College praise the College Board’s claim of the be due not to the AP classes
Entrance Examination Board (College equivalence of Advanced Placement themselves, but to the personal
Board) to “allow participating col- courses as measured by AP exam characteristics that led them
leges and universities to award high scores with the comparable college to participate in the classes
school students college credit, ad- course in three science disciplines: in the first place – better
vanced placement, or both” (College biology, chemistry, and physics. This academic preparation, stronger
Board, 2004, p. 6). While the granting is a complex issue because, to some motivation, better family
of credit or placement remains at the degree, AP course-taking is a sorting advantages, and so on. These
discretion of each individual college process that draws primarily highly selection effects will affect any
or university, the College Board is motivated, high achieving students. comparison of AP and non-AP
unequivocal in its interpretation of If this self-selection effect is ignored, students (p. 3).
AP exam scores:
Colleges vary considerably in their
AP Exam grades of 5 are policies and in their advice to their
equivalent to the top A-level Colleges vary considerably students concerning enrollment in
work in the corresponding in their policies and introductory science courses. As a
college course. AP Exam grades in their advice to their result, there exists a sizable population
of 4 are equivalent to a range students concerning of students taking introductory science
of work representing mid-level
enrollment in introductory courses who have done well in AP
A to mid-level B performance
science courses while in high school,
science courses.
in college. Similarly, AP Exam
. In this research we refer to the numerical score earned on the Advanced Placement exam (, 2, 3, 4, or 5) as the “AP exam score.” We refer to the teacher’s as-
sessment of the AP students’ coursework as the “AP course grade.” In its literature, the College Board generally refers to the AP exam score as the “AP grade.”
Fall 2007 Vol. 16, No. 2 
yet do not “place out” in college. Since With tests scores in hand, students
the AP exam is taken prior to college are assured by the College Board
High school AP courses
entry, one would expect that these that “Over 90 percent of the nation’s
corresponding to the 35 AP
students, whom the College Board has colleges and universities have an AP
examinations administered
awarded passing exam scores, would policy granting incoming students
earn substantially higher grades, on the by the College Board in both credit, placement, or both, for qualify-
average, upon re-taking an introduc- science and non-science ing AP Exam grades.”7 However, the
tory college course in the same sub- disciplines are taught by fine print is more complex. Having
ject. The apparent advantage is clear. 110,000 teachers in the US. an “AP policy” does not mean that
Students exhibiting higher levels of colleges and universities follow the
AP performance would most certainly College Board’s recommendation that
and offer a means of certifying each
be expected to have a stronger grasp an AP score of 3 should garner course
student’s mastery of course content at
of the relevant college content. They credit or advanced placement. Many
the college level. In the sciences, stu-
should do better than students who times, colleges only offer credit for
dents sat for a total of 3,595 exams
have not had the advantage of what scores of 4 or even 5. Some college
in environmental science, biology,
is typically a second year of study in policies do not grant either to incom-
the subject while in high school. The chemistry, or physics in 2005.3 Among ing students.8
these science test-takers, 59% earned
intent of this paper is to characterize the While the College Board puts much
scores of 3 or higher (5-point scale).
performance of these students as both energy into the construction, monitor-
By comparison, 62% of mathematics
an independent approach to the valida- ing, and scoring of AP exams in an
test-takers earned scores of 3 or better.
tion of Advanced Placement exams in attempt to attain a measure of college
As a result, many of these students
biology, chemistry, and physics and as course equivalency, there is a consider-
would have grounds to expect their
an estimate of the contribution of these able range of opinion concerning the
prospective colleges to offer them
advanced high school courses. value of AP courses in the sciences and
course credit in these disciplines when
mathematics. Many professors believe
Background they matriculate.4 This outcome would
that all students can benefit from taking
The AP Program of the College be consistent with the original purpose introductory science courses in college
Board provides “motivated high of Advanced Placement envisioned (National Research Council, 2002, p.
school students with the opportunity by representatives of the faculties of 58). Others argue that college-level
to take college-level courses in a high the three private academies (Andover, courses require the efforts of college-
school setting.”2 High school AP Exeter, and Lawrenceville) and level faculty (Juillerat, Dubowsky,
courses corresponding to the 35 AP three private universities (Harvard, Ridenour, McIntosh & Caprio, 997).
examinations administered by the Princeton, and Yale) responsible for Some colleges administer their own
College Board in both science and formulating the program.5 To promote placement exams. Others allow credit
non-science disciplines are taught by this policy, the College Board makes only for students not majoring in that
0,000 teachers in the US. These available research studies that support specific field or award credit only after
exams are central to the AP Program its contentions.6 a student has taken and passed a higher-
2. College Board AP Central website. http://apcentral.collegeboard.com.
3. http://apcentral.collegeboard.com/members/article/,3046,52-7-0-39036,00.html
4. “ACE [American Council on Education] notes that the practice of granting provisional credit for a grade of 2 is becoming more frequent because comparabil-
ity studies have shown that many of these students are potentially qualified for college-level work -- credit is made permanent if the student satisfies another
criterion, such as successfully completing the next course in the discipline.” http://apcentral.collegeboard.com/apc/public/colleges/setting_policy/index.html
5. “[For] the student of marked mathematical and scientific ability and interest, who is probably heading for a college major in science and a career in science or
engineering … could take a one-and-a-half or two-year school course in chemistry, physics, or biology, designed to prepare him for admission to the sopho-
more college course in the field chosen for concentration in school [i.e. in college]. (Phillips Academy, 1952, p. 65.)
6. http://apcentral.collegeboard.com/article/0,3045,52-67-0-592,00.html
However, research that can be considered somewhat critical of the claims of the College Board concerning the AP program does not appear to be included at
their website (e.g. Geiser & Santelices, 2004; Lichten, 2000).
7. http://apcentral.collegeboard.com/program/ 0,3060,50-0-0-0,00.html
8. Lichten’s (2007) sampling of colleges and universities found that only 30% of colleges accept an AP exam score of 3 for advanced placement, while 89%
accept a score of 4, and 95% accept a score of 5.
2 ScieNce educator
level course in the subject (National addition to attracting potentially more lowest score of  (out of 5) on the AP
Research Council, 2002, p. 58). Some highly motivated and gifted students, exam have not demonstrated content
universities offer special transitional there are other potential covariates: mastery. However, this outcome leads
courses for those students who place AP courses are more prevalent in one to wonder whether this year for
out of introductory courses, but do wealthier communities, are typically these students of study could have been
not advance to the next level course taught by more experienced teachers better spent in a course with a differ-
(National Research Council, 2002, p. (Burdman, 2000) and predominantly ent format. Also troubling are the low
59).9 Lichen (2000) argues that the enroll students who have already AP science exam passing rates among
quality of AP courses has diminished studied the subject for a year. Much under-represented groups.2
to the point that the College Board’s of the existing AP research gives no
Prior Research
recommendations are no longer ac- recognition to these factors. Studies
cepted by a large number of colleges, lacking controls suffer the potential Research on the value of AP
particularly those considered highly weakness of overestimating the effects courses in the sciences has primarily
selective (e.g. Harvard, Yale, MIT, of AP courses, because they conflate focused on accounting for differences
University of Pennsylvania).0 An the influence of numerous factors and in freshman grade point average or
NRC committee recommends that col- fail to account for the relationship of persistence to graduation, both rather
leges should be strongly discouraged AP courses net of the covariates. While indirect measures of the equivalence
“…from using college scores on AP an many assume that students who do of AP courses and introductory col-
IB exams as the sole basis for granting well in AP courses in high school owe lege science courses. Others judge the
automatic advanced placement out of their success to their AP studies, other performance of students who “place
specific courses for majors, or out of factors may well be involved, such as out of” one or two semesters of the
biology distribution requirements for coming from more advantaged homes introductory course in a scientific field
non majors (Wood 2002, p. 25.)” and attending schools that emphasize and then enroll in a second or third
The population of students who college preparation (Willingham & semester science course equivalent
have taken AP courses in high school Morris 986). Moreover, the fast pace to students who take the prerequisite
and retake introductory courses has of AP courses, due in large measure course in college. Such studies have
been has been largely neglected by re- to the focus on test preparation, may not controlled fully for the fact that
searchers. Surprisingly, researchers not serve students as well as courses AP students generally have stronger
have rarely structured past studies to that progress at a more moderate pace, high school preparation in math and
reveal the degree to which AP courses focusing on topics in greater depth and English, contributing to the ability to
bestow an added benefit upon students detail. This supposition led a recent na- “skip ahead.” Moreover, second and
who take them, “… the existing em- tional study to posit that, “It is possible third semester science courses can
pirical evidence regarding the benefits that AP students were at a disadvantage cover content unrelated to the pre-
of AP experience is questionable in some classes or at some colleges” requisite course (e.g. second semester
(Klopfenstein and Thomas, 2005, p. (National Research Council, 2002, p. physics usually entails a study of elec-
5).” Klopfenstein and Thomas refer to 94). While most assume that such an tricity, magnetism, and light, topics
the body of AP research, a majority of outcome is unlikely, we are willing to distant from first semester kinematics
which lack or include insufficient con- subject such hypotheses to scrutiny. and Newtonian mechanics). In most
trols. We should keep in mind that, in For example, students reporting the cases, studies examine the perfor-
9. None of the courses participating in this study have that profile.
0. USA Today, 3/20/2006, Advanced Placement: A detour for college fast track? Mary Beth Marklein
. Studies comparing student performance in second-year college courses typically compare those who have “placed out” with high AP exam scores to those
with students who take the introductory level college course with either a failing AP exam score or no AP experience (Dodd, Fitzpatrick, De Ayala, & Jen-
nings, 2002). No mention is found in the literature of the experience of those students who chose to retake the introductory college course with a passing AP
exam score.
2. Fewer than half of under-represented minorities are “passing” the AP exam with scores of 3 or higher, while scores of  (the lowest score possible) are
awarded to more than half of African American and Chicano students. Put in another way, students not “passing” the exam constitute 70% of Chicano and
African-American students, while more than 50% of Native Americans and Puerto Rican students score below 3. Based on this measure, AP courses do not
appear to be an effective means of AP exam preparation for many students. For more information, please consult the AP National Report 2005, http://www.
collegeboard.com/student/testing/ap/exgrd_sum/2005.html
Fall 2007 Vol. 16, No. 2 3
Shellhammer, 978). Willingham difference was reported to be statisti-
and Morris’ (986) found that cally significant for one year of the
The élan of having AP
about half of the grade differen- study. The AP-Class group earned
courses, especially in
tial (those with a college grade slighted higher grades in the second
science and mathematics,
average of B or above) of AP semester course than the AP-CR
is well known.
students over non-AP students group. Reported GPA’s in other college
was accounted for by matching biology courses were also higher for
pairs of 878 students on their Non-AP students.
mance of students at a single college
or university or within a single state, academic background. Klopfen- • Persistence to college gradua-
making generalization to the entire stein & Thomas (2005) found that tion – The combined number of
U.S. problematic. These studies also of 28,67 students, those taking AP and honors courses taken in
fall into two general categories: studies AP courses in high school did high school was not a significant
with adequate control variables and not earn higher freshman grades predictor for persistence through
those without. The latter group typi- in college. freshman to sophomore years in
cally compared grand means between • College Science Course Grades. college (Geiser & Santelices,
samples of AP and non-AP students or Ruch (1968) did not find a sta- 2004); Klopfenstein & Thomas
correlated the performance of AP stu- tistically significant difference (2005) found students taking AP
dents with other performance-related between matched pairs of AP courses in high school have no
variables. We found that these studies and non-AP students on college higher probability of persisting
typically appear as reports that are not grade in the courses related to the to a second year of college. Low-
published in peer-reviewed journals. AP course taken or overall fresh- income students who take AP
Often the source of funding for much man GPA. Dodd, Fitzpatrick, courses in high school graduate
of this work is unclear. On the other and Jennings (2002) studied the college at a greater rate (Dough-
hand, the “with controls” studies more performance of three groups of erty, Mellor & Shuling, 2006a)
frequently appear in peer-reviewed 83 students in a second semester • Choice of Further Study – Ruch
journals, having been subjected to biology course at the University (968) used 2 matched-pairs
more scrutiny and review. of Texas at Austin. They compare of students and found that that
While we reviewed both forms of the mean grades of three student AP students were more likely to
research, we summarized only the groupings: continue with the same subject
findings of the “with-controls” studies . Those who earned credit for the in college. Chamberlain, Pugh,
below, grouping the findings under prerequisite course with a score & Shellhammer (978) found
four commonly analyzed outcome of 3 or above on their AP exam academic progress, college GPA,
variables. The 986 Willingham and (AP-CR). and course-taking measures were
Morris report was especially useful in superior for AP students over 344
2. Those who earned lower than
constructing this list. a 3 in their AP course and took matched sets of students (Cahow,
• College Grade Point Average the prerequisite (AP-Class). Christensen, Gregg, Nathans,
(GPA) – The total number of AP Strobel, & Williams, 979). Will-
3. Those who did not take an AP
courses taken in high school has ingham and Morris’ (986) found
class and enrolled in the perqui-
little effect on predicting fresh- that of 878 students, AP students
site class (Non-AP).
man college GPA, but AP exam were no more likely to take more
scores are a strong predictor of AP-CR and Non-AP groups were than one year’s coursework in
sophomore grades (Geiser & matched using high school class rank biological or physical science,
Santelices, 2004). Using matched and SAT and ACT scores. Over a nor to major in those fields.
sets totaling 688 students at four-year period, the Non-AP group
All of the studies above controlled
Indiana University, AP students earned higher mean grades than either
for students’ high school performance
were found to earn higher college the AP-Class or the AP-CR groups
(i.e. rank, HSGPA, or coursework
GPA’s (Chamberlain, Pugh, & in the second semester course. This
4 ScieNce educator
completed) except for Chamberlain in order to be considered robust. For 2. million entering freshmen enrolled
et al. (978) and Dougherty et al. example, inclusion of SAT scores in degree granting institutions in Fall
(2006a). Geiser & Santelices (2004), alone, entered as a control for high 2003. One might assume that on the
Klopfenstein & Thomas (2005), and school-level learning, is insufficient average nearly half of all entering col-
Dougherty et al. (2006a) included to account for prior science knowl- lege freshmen have taken at least one
measures of students’ socio-economic edge learned in first-year high school AP course in high school.4 In reality,
status. Ruch (968), Dougherty et al. science courses. It is incumbent on many have taken more than one course.
(2006a) and Klopfenstein & Thomas the scholar employing regression to Our estimate is that in 2004, those
(2005) included high school quality explore alternative relationships that taking AP science exams represented
ratings, while Willingham & Morris are potentially causal and to take great one of five college freshmen (Digest
(986) matched students by college care in considering, measuring, and of Educational Statistics, 2004).5
attended. including factors that could account The expansion of AP Programs is
From this short list of controlled for additional variance. Put another also currently propelled by economic
studies, it is apparent that the measure way, it is the attention that scholars and political support since several
of AP coursework value varies with the pay to controlling for alternative states currently extend financial in-
outcome variable considered and the hypotheses that is often a measure of centives to high schools that offer AP
control variables included. We did not quality research. courses6, while others require all high
find any studies that measured perfor- But, why is research on the direct schools to participate (Hershey, 990;
mance in introductory college science comparison of AP and non-AP students Willingham & Morris, 986).7 Many
courses while controlling for relevant important? Conventional wisdom states have mandated through legisla-
covariates such as performance in high ascribes clear and overwhelming ben- tion the inclusion of AP enrollment in
school courses considered prerequi- efits to Advanced Placement classes admissions decisions or the granting
sites for AP science courses. Given in high schools. The élan of having of college credit for high AP exam
that performance in prior coursework AP courses, especially in science and scores in public institutions of higher
and is widely regarded as the strongest mathematics, is well known. As a re- education (Lichten, 2000). The U.S.
predictor of subsequent performance, sult, the College Board’s AP program president has called for an addition of
it is essential that this factor should has grown dramatically over the last 70,000 Advanced Placement teachers
be included in any analysis seek- 20 years, with a sustained increase in science and mathematics, a near
ing to isolate the relationship to AP in the number of exams including all doubling of current numbers.8 In addi-
coursework itself, rather than merely subject areas of 9.3% (with an 8.8% tion, the U.S. Department of Education
considering an aggregated analysis increase in number of students) annu- subsidizes AP programs for low-in-
that lacks both detail and is unable ally. To offer some perspective, this come students (Klopfenstein, 2004).9
to estimate the relative contributions growth is more than five times faster This expenditure of public funds, in
of various factors. The inclusion of than the growth in the US high school effect, subsidizes one particular edu-
such controls provides an opportunity population.3 The ratio of students to cational program over others. Hence, it
for refinement of models that more exams tracked closely until recently; is desirable that Advanced Placement
accurately estimate the relationship their separation is indicative of an policies and programs be rigorously
of AP coursework relative to other increase in the number of students tak- assessed so that scarce resources can
factors. Regression models must be ing multiple AP exams (.29 in 970 be spent the most wisely.
constructed using variables represent- vs. .72 in 2005). In 2005, .2 million A powerful motivating factor for
ing viable alternative hypotheses to students took 2. million AP exams. students is the fact that one may enter
the main relationship being studied According to the latest available data, college with course credits and in some
3. http://apcentral.collegeboard.com/members/article/,3046,52-7-0-47040,00.html.
4. http://nces.ed.gov/programs/digest/d05/tables/dt05_80.asp.
5. http://apcentral.collegeboard.com/members/article/,3046,52-7-0-39036,00.html.
6. Florida, Louisiana, and Utah.
7. South Carolina.
8. George W. Bush, State of the Union Address, /3/2006.
9. Higher Education Act Amendments of 998, Title VIII, Part B, P. L. 05-244.
Fall 2007 Vol. 16, No. 2 5
cases with enough credits to bypass the analyzed survey data gathered from of participants, provided they are from
freshman year, saving a year’s tuition students in 124 different first semester a well-selected random sample.
(MacVicar, 988; Pushkin, 995). introductory college biology, chemis- Choosing to focus only on under-
Troubling to many professors is that try, and physics courses in 55 colleges graduates, we omitted cases of 03
students may use AP credit in many and universities. These institutions graduates and 400 non-traditional
colleges to satisfy their requirements were from a stratified random sample students. We also omitted 397 students
for science courses and never take reflecting a nationally representative who spent their high school years
another science course when in college distribution of institutional size and outside of the United States. Allowing
(National Research Council, 2002, selectivity, from small liberal arts for some students in the intersection of
p. 59). For example, Lichten (2000) colleges to large state universities. these various groups, our final sample
found that only 22% of students who All participating introductory science included 8594 undergraduates.
had scored “3” on their AP calculus courses met in lecture-style classes Of this total, 3% of the sample was
exam enrolled in an advanced calculus with weekly recitations sessions and enrolled in biology, 42% in chemistry,
course in college, while 7% took a laboratories and had enrollments of and 27% in physics. Females made up
remedial calculus course. Though 0 students or more. 60% of the biology students, 55% of
high school AP teachers may be very Of 9354 collected surveys in this the chemistry, and 42% of the physics
talented, they generally do not have subsample, 029 respondents reported students. Parental education was relat-
the breadth of knowledge nor the most having taken an AP course in the sub- ed to the highest level of mathematics
recent and relevant content knowledge ject in high school in which they were taken by students, with representation
possessed by college faculty (Lichten currently enrolled in college. Of the of students taking calculus in high
2000). By accepting AP exam-related students who took AP courses, 36 school ranging from 33% of student
course credits, students may be forego- earned scores of 3 or higher on the with parents having a high school di-
ing the opportunity to learn about the AP exam, roughly 3% of the entire ploma or less to 49% of students with a
most topical findings and issues from sample. While this can be considered parent holding a graduate degree. The
college instructors whose “other” job a small number compared to the sample identified themselves as 77%
it is to do research. 70,000 students who earned scores White, 7% Asian/Pacific Islander, 6%
The striking expansion of the AP of 3 or higher on AP exams in these black, 5% Hispanic, 3% multi-racial,
program offers an opportunity to subjects in 2005, the data used in this and 2% other or unreported.
gauge the pedagogical influence of This study, by its very nature, may
AP coursework. Merely presuming only examine students who take intro-
that AP courses are comparable with … many students feel that while ductory-level college science courses.
college courses ignores the opportu- AP is a good preparation for Students enrolled in colleges that offer
nity we have to analyze the degree of college science, many also feel the option of bypassing introductory-
this purported similarity as a means to level science courses with sufficient
they benefited from taking the
improve college preparation through AP exam scores and who also make
actual college science course.
high school instruction. the decision to take advantage of this
option to place out are not included.
Methods
analysis represents a national sample Yet, since prior research studies have
The Sample of students who have both taken part generally ignored this interesting
Factors Influencing College Science in Advanced Placement courses and population, studying these students
Success (Project FICSS) is a large- exams in high school and also take can make a valuable contribution to
scale national study of introductory introductory level college science understanding the value of Advanced
college students and their high school courses. The theory and practice of Placement programs.
science experiences, which collected a inferential statistics has established Students who did poorly on AP
total of nearly 8,000 surveys. The data that valid inferences (within ap- exams in a given science discipline
includes a series of subsamples of col- propriate confidence limits) can be may choose not to take the college
lege students in various college science drawn from relatively small number course. By comparing the national

courses. In this particular analysis, we statistics for AP exam scores with the
6 ScieNce educator
AP Exams and College Performance in Science
FFiigguurree 1 1. :D Disitsrtirbibuutitoionn o off A APP  EExxaamm  SSccooreress i nin S Samampplele v vss. .N Naatitoionnaal lN Noorrmmss
40%
35%
30%
25%
Sample
20%
National Norms
15%
10%
5%
Error Bars ±1 SE
0%
AP 1 AP 2 AP 3 AP 4 AP 5
We were also careful to check for any anomalies in the makeup of students who
distribution in our sample, one can see a relative dearth of students earning Placement exam in the discipline of
earned scores of 3, 4, or 5 on their AP exams. Table 1 displays a comparison across
differences (Figure ).20 The five AP low AP exam scores. their introductory college science
seven measures of academic performance among nine high school science preparation
score categories each average about We were also careful to check for course and received a score of 5.22 In
categories ranging from Computed grades represent the mean grade for a particular
one-fifth of the students who take the any anomalies in the makeup of stu- general, AP students outperformed
group (on a 100-point scale where A=95, B=85, etc.). “Not Taken” for students reporting
exam.2 It appears that students at the dents who earned scores of 3, 4, or 5 honors students, who in turn outper-
that they had not taken a high school course in the discipline of their introductory
low encdo l(lAegPe e xscaimen scceo rceos uorfs e atnod “ 2A) P 5”o nfo trh estiru AdePn etxsa rmeps.o Trtaibnlge th deiysp hlaayds takefonr mtheed A redgvualanrc setdud ents, who in turn
and atP tlhaec heimghe netn de x(aAmP einxa tmh es cdoirsecsi plinae c oofm tphaeriirs oinnt arocrdoussc tsoervye nc omlleeagsue rsecsi encoeu ctpoeurrfoserm aendd students with no high
of 4 raencde i5v)e dar ae sucnodreer -orfe p5r.2e2s Ienn tgede neraolf, AacPa sdteumdiecn tpse orfuotrpmearfnocrem aemd ohnogn orss cshtuodole cnotusr, swewhoor kin in the correspond-
in thet usarnm poluet. pPerrefsourmmaebdly r eag lualragre rs tudneinntes h, iwghh osc ihno otul rsnci eonuctep perrefoparmrateiodn s tudienng tcso wlleigthe sncoie hncige hd iscipline. Within
fractiosnc hoof othlo csoeu wrhseow scoorrke di nb etlhoew c 3o rrescpaotengdoirniegs croalnlgeigneg scfrioemnc eC doimscpiuptleidn e. Wthiet hAiPn gtrhoeu pA, Pw e see a clear pattern of
chose gnroot utop e,n wroell sine ea cao cllleegaer cpoautrtseer.n ofg hraigdhese rre pperersfeonrtm thaen mcee acno rgrreasdpe ofonrd ai ng thoi ghhiegrh peerr fAorPm eaxnacme c orresponding to
scores. High scoring AP 4 and AP 5 students consistently reported higher SAT scores,
Many at the higher end arguably took particular group (on a 00-point scale higher AP exam scores. High scoring
mathematics grades, in-discipline science grades for regular or honors courses, and
advantage of placement to enter a where A=95, B=85, etc.). “Not Taken” AP 4 and AP 5 students consistently
grades in other science courses. Higher scoring students in our sample also reported
higher-level course or received credit for students reporting that they had reported higher SAT scores, math-
higher grades in their AP courses, on average. Students in our sample who chose not to
for the introductory course. The mean not taken a high school course in the ematics grades, in-discipline science
AP exam score for the national data is discipline of their introductory college grades for regular or honors courses,
slightly lower (2.8) than the sample science course to “AP 5” for students and grades in other science courses.

used in this analysis (2.99) reflecting reporting they had taken the Advanced Higher scoring students in our sample
22 Regular signifies that the highest level of course taken in high school was neither AP nor honors level.
Honors signifies an honors course. AP courses are broken down by AP exam scores of 1 through 5 or the
20. The cnaattieognoalr syta otifs tisctsu fdore AnPts b nioolotg tya, kchinemgi satnry ,A anPd epxhyasmicsin haavtieo bne.e nM aevearnagse adr.e reported for the SAT math or SAT verbal
2. Ones schooruelds n(ootre tthhaet tehqe udiisvtraibleuntiot nf roof msco trhese f oAr CthTe )A, Pt hexea fmr aisc tnioot nG acuosmsiapnl aest iins rge pao rctaedlc fuorl umso scto nuatrisoen ailn s thanigdahrd sizcehdo eoxla,m tsh seu lcahs ats the SAT or ACT.
Whigler athdoese iant tmhea ctehnetemr oaft tihces ,d itshtreib mutieoann c agnr baed teh oiung htht oef raes leeavrnainngt cslcoiseen toc eth pe rmeepaanr sactoorer,y th cooseu arts teh e( i4f lteavkele onr) a, btohvee m(35e%an) or at the 5 level (6%)
occugr rwaidthe m inuc thh ger eAatPer cfroeuquresnec y(i tfh taank setund)e,n atsn adt o tnhee s taavndearradg dee voiaft isocni eabnocvee g(r6a%d)e osr i tnw ofi setladndsa ordt hdeevri athtioanns athboev ce o(2ll%eg).e science
22. Regcuolaur srisgen iofife si nthtaetr tehse th. ighest level of course taken in high school was neither AP nor honors level. Honors signifies an honors course. AP courses are
broken down by AP exam scores of  through 5 or the category of students not taking an AP examination. Means are reported for the SAT math or SAT verbal
scores (or the equivalent from the ACT), the fraction completing a calculus course in high school, the last grade in mathematics, the mean grade in the relevant
scienScaed plreerp aarnatdor yT caoiu rse (if taken), the mDearna gfrta 8de/ i1n3 t/he0 A7 P— co Dursoe N(ifo tat kDenu),p alnicda thtee aovre rQague ootfe s c ience grades in fields otherP thaagne t h1e0 c ollege science
course of interest.
Fall 2007 Vol. 16, No. 2 7
also reported higher grades in their AP various high school science courses students in the field or their academic
courses, on average. Students in our including AP, regular, or honors). On advisors. Five took the course because
sample who chose not to take the AP the contrary, we found in our sample of department requirements in their
exam, even after taking the AP course, that students who reported an AP disciplines. Three reported AP exam
reported academic performance mea- exam score of 5 also averaged 300 scores of 4, while their college required
sures similar to the students in the AP on their SAT’s, with three-quarters a score of 5 for credit. Two reported
3 category. having taken calculus in high school, low scores on departmental placement
In our analysis, we were particu- and also reported earning an “A” on exams required by their colleges to
larly concerned with the academic average in their high school science earn course credit. All but one felt
performance in college of students who and mathematics courses. that their AP course prepared them
reported receiving AP Exam scores of Who are the group of students who well for introductory college science.
3, 4, or 5. We wondered if the students have taken an AP course in high school However, eleven reported that taking
in our sample were particularly weak in and have earned an AP exam score the introductory college course as a
these science disciplines despite theiAr P Egxarematse ar nthda nC 3ol?l eWgeh yP aerrefo trhmeya tnackei ning Sanci encevery worthwhile experience that paid
high AP exam scores. Might a weak- introductory course in college? Were off for them later in their college sci-
ness in their academic background they advised or required to take this ence course work. Four noted, in ret-
have lteadk eth ethme tAo Pp uerxsuaem a,d edviteionn aalf ter ctoaukrisne gby t thheei rA coPl lceogeu?r Tsoe g, arienp a obrettteedr acarodsepmecit,c a fpteerr tfaokrinmg aadnvcaen ced courses,
coursem weoarksu art etsh es iimntrioladru cttoo rtyh eco sl-tudeunntdse risnta tnhdein Ag oPf p3o scsaibteleg moorytiv. a tions, that they should have moved directly
lege level? We imagined that students we contacted 44 students who both to higher-level course work.
In our analysis, we were particularly concerned with the academic performance in
with high AP exams, but relatively scored well on AP exams and took As one of those students noted:
college of students who reported receiving AP Exam scores of 3, 4, or 5. We wondered if
weak tahcea dsetumdice nbtasc king roouunrd ssa mmapyl e wienrtreo pduacrttoircyu claorllleyg ew secaiekn cien ctohuerssees s cienc•e Id ivsecriyp mlinucehs dsheosupldit eh ave been
not bet fhuelliyr flhuiegnht iAn PE negxliasmh, osrc moraeys . Minig thhets ea d wisceiapklinneess. sT hines eth steuidre antcsa hdade mic bac[kggrarnotuedn]d c hreadvite. Tlehde class was
have wtheeakmer t om patuh rbsaucek gardoudnitdiso, noar l covuorlsuen tweeorerdk t haet irt hcoen itnactrt oindfourcmtoatriyon c ollege lpervetetyl? m Wuceh iam waagstien oefd m y time.
may have earned lower overall high in response to a question asking if they
that students with high AP exams, but relatively weak academic backgrounds may not
In contrast, others offered more
school grades. To address these pos- would agree to future contacts in light
be fully fluent in English, or may have weaker math backgrounds, or may have earned
positive reports of their college science
sibilities, we analyzed these various of survey related queries. Among the
lower overall high school grades. To address these possibilities, we analyzed these
course experiences:
background measures and did not find responses, fifteen students provided
various background measures and did not find that, compared with other students,
that, compared with other students, detailed replies. While these responses • It was worthwhile and I am glad
high AP exam scorers exhibited any weaknesses in language proficiency (SAT Verbal
high AP exam scorers exhibited any should not be considered representa- I chose to take the intro level
score), mathematics proficiency (SAT Math score, high calculus course taking patterns,
weaknesses in language proficiency tive, they do offer some insight into course. I recommend it to fresh-
high school mathematics course grades), or overall high school grades (grades in
(SAT Verbal score), mathematics the diversity of students’ experiences man coming in who face a similar
various high school science courses including AP, regular, or honors). On the contrary,
proficiency (SAT Math score, high and opinions. Five made the personal situation. Taking the introductory
we found in our sample that students who reported an AP exam score of 5 also
calculus course taking patterns, high decision to take the introductory course course refreshes my memory on
averaged 1300 on their SAT’s, with three-quarters having taken calculus in high school,
school mathematics course grades), or even though they could have placed the subject, provides deeper
and also reported earning an “A” on average in their high school science and
overall high school grades (grades in out, citing advice from more advanced insight into the subject that just
mathematics courses.
TTaabbllee 1 1: . CCoommppaarirsioson no fo Hf iHghi gShc hSocohl oPoelr fPoermrfaonrcmea Mnceea sMureeass buyr eGsr obuyp Group
Highest HS Other Grade in
Course or AP SAT HS Last Math Science Honors or
Score Sat Math Verbal Calculus Grade Grades Regular AP Grade
not taken 475 454 27% 86.0 88.9
regular 529 510 32% 87.7 89.4 88.8
honors 596 564 60% 89.4 90.8 90.4
AP, no exam 610 574 70% 89.6 91.1 92.4 87.6
AP 1 543 508 47% 85.2 88.4 90.0 85.0
AP 2 592 541 54% 90.1 90.5 91.5 88.1
AP 3 595 568 61% 90.1 90.9 91.8 88.9
AP 4 631 628 64% 91.0 92.1 93.9 92.2
AP 5 666 654 75% 92.5 93.0 94.0 94.1
Who are the group of students who have taken an AP course in high school and have
8 earned an AP exam score greater than 3? Why are they taking an introductory coSucireNscee iend ucator
college? Were they advised or required to take this course by their college? To gain a
better understanding of possible motivations, we contacted 44 students who both
scored well on AP exams and took introductory college science courses in these
disciplines. These students had volunteered their contact information in response to a
question asking if they would agree to future contacts in light of survey related queries.
Among the responses, fifteen students provided detailed replies. While these responses
should not be considered representative, they do offer some insight into the diversity of
students’ experiences and opinions. Five made the personal decision to take the
introductory course even though they could have placed out, citing advice from more
advanced students in the field or their academic advisors. Five took the course because
of department requirements in their disciplines. Three reported AP exam scores of 4,
while their college required a score of 5 for credit. Two reported low scores on
departmental placement exams required by their colleges to earn course credit. All but
one felt that their AP course prepared them well for introductory college science.
Sadler and Tai Draft 8/13/07 — Do Not Duplicate or Quote Page 11
AP Exams and College Performance in Science
Figure 2. Survey Questions Relating to High School Course Taking in Science and
Figure 2: Survey Questions Relating to High School Course Taking in Science and 
Mathematics
Mathematics
Two major concerns loom for any large-scale self-report study, accuracy and reliability.
These concerns stem from a report by Bradburn, Rips, and Shevell (1987) that appeared
wasn't there in high school, and to complete these surveys while in their and unreliable. However, in research
in the journal Science. The study concluded that self-report data were inaccurate and
prepares me for higher level college classes. Using the names and that has followed in the intervening
unreliable. However, in research that has followed in the intervening years, which
courses in the subject. ID numbers provided by students on years, which includes further research
includes further research by Bradburn himself, these conclusions have been qualified
• It was definitely a worthwhile the cover, college instructors entered by Bradburn himself, these conclu-
and mollified to portray a complex and more responsible conclusion: that the accuracy
experience to retake the classes, students’ course grades. After this sions have been qualified and mollified
and reliability of self-report likely depend on several factors including context,
as it gives you a second chance process was completed, the perforated to portray a complex and more respon-
relevance, and survey clarity Pace, Barahona & Kaplan, 1985; Bradburn, 2000; Niemi &
to absorb the material—which cover with student identification was sible conclusion: that the accuracy and
Smith, 2003). In particular, self-reports of course taking, grades earned, and
can never hurt. torn away and discarded, leaving the reliability of self-report likely depend
standardized test scores made by college students tend to be highly accurate (Baird,
surveys anonymous, except for any on several factors including context,
• I was a1t9 l7ea6s; tA fanmailyiaar, w19it9h 9m).a nEyn rollment reports are especially accurate for courses with
contact information students may have relevance, and survey clarity Pace,
of the utonpaicms bcoigveuroedu sin n maym ceosl -and for high-achieving students (Sawyer, Laing & Houston, 1989).
volunteered in response to a “further Barahona & Kaplan, 985; Bradburn,
lege cIlans sa. rMeacneyn tc ornecveipetws w oefr ee xisting research on self-report, Kuncel, Credé, & Thomas (2005)
contact” request. Students also re- 2000; Niemi & Smith, 2003). In par-
coverecdo nwcilthu dgere tahteart sdeeltfa-irl eipno rt may be characterized as reasonably accurate in samples where
ported on a variety of demographic ticular, self-reports of course taking,
collegteh. eI fseultr vweeyll sp aredpdarreeds sf oirs sues relevant to the respondents. Our study surveyed
variables (parents’ education, profes- grades earned, and standardized test
the coillnetgreo cdlauscst, obruyt Ic aomll gelgaed sI cience students in their fall semester college science classes where
sion, etc). Home ZIP codes allowed scores made by college students tend
took three fclleacstsi oinns toeand t hofe tiers ptinrigo r experience would be commonplace. In addition, the students’
us to match the surveys with existing to be highly accurate (Baird, 976;
out. own instructors administered the surveys.
data on median income, home value, Anaya, 999). Enrollment reports are
The stuWdeen te ncotemrmeden tths isin pdircoajteec t wainthd apvreervagioe ueds uscuartivoenayl rbeascekagrrocuhn de xpeersipeencciael liyn awcchuircahte wfoer scuourrvseeys ewdit h
that both mootviveart i2o0n0s 0an cdo elxlepgereie pncheyss icso sft uedacehn thso imne o vloecra l2e0. dTihfef esruernvte cyo lleugnea manbdig uuonuisv enrasmiteise sa n(Sd afdolre rh i&gh -
associated Twaitih, 2in0t0ro1d).u Tcthorey dceosllieggne of tqhuee sctiuornrse tnhte msuserlvveeys winecrelu cdareedfu tlhlye usaec hoife vstinugd setundte fnotsc u(Ssa gwryoeur, pLsa itnog &
science coucrosmes mdifefnert woind eqlyu.estion fordmesaitgsn,e tdw too fipti loont tshue rfvroenyt sa nthd abta icnkc ludHeodu s3to0n4, co9l8l9e)g. eI ns cai ernecceen t review
students, a review by an aodf vthiseo rreym paianninegl othf rceoe llsehgeeet sp. rSoefee ssorosf aenxidst ihnigg hre ssecahrocho l otne ascehlfe-rresp, ort,
The Survey
and a separate reliability Fstiguudrey 2w. here 113 college chemistryK sutnucdele, nCtsre tdoéo, k& t hTeh soumravse y(2 005)
The survey instrument was con-
twice, two weeks apart andT wwho emraej othr ecoirn creersnps olonosmes f owr earney comcopnacrleudde. Mthuatc hs eclaf-nre bpeo rdt omnaey t ob e
structed to collect data concerning aca-
enhance accuracy and reclaarlgl:e c-sacraelef usell fd-reespiogrnt s, tcuodyn,t aecxctuuraacly c uesc,h aarnadct epriazretdi caisp raenast ornealbelvya anccceu rate
demic experiences and performance
all play important roles. and reliability. These concerns stem in samples where the surveys address
measures closely associated with high
from a report by Bradburn, Rips, and issues relevant to the respondents. Our
school scieInnce l caorugres-es wcaorlke tshuatr mveigyhst, missing responses are not uncommon. In dealing with missing
Shevell (987) that appeared in the study surveyed introductory college
predict perrfeosrmpoanncsee sin, liinsttr-owduisceto drye letion is the simplest and most commonly applied option.
journal Science. The study concluded science students in their fall semester
college scieHncoew cleavsseers,. lIins tth-we fiosrem d oef letion assumes that the data are missing at random. Should a
that self-report data were inaccurate college science classes where reflec-
a 4-sheet boloakrgleet, sptuedrceentnst waegree aosfk ethd e sample exhibit missing responses, this approach may not be the
Fall 2007 Vol. 16, No. 2 9
Sadler and Tai Draft 8/13/07 — Do Not Duplicate or Quote Page 13
tion on their prior experience would ing responses and found that, for the AP course should be included. Table
be commonplace. In addition, the variables of concern, only about -2% 2 shows the Pearson correlations be-
students’ own instructors administered were missing for each. Given this tween academic preparation variables
the surveys. characteristic, we chose to use list-wise and college course grade.
We entered this project with pre- deletion. The two exceptions were Among the nine academic perfor-
vious survey research experience in that no standardize test score (SAT mance measures, AP Score exhibits
which we surveyed over 2000 college or ACT) was entered for 347 cases the highest correlation with College
physics students in over 20 different (quantitative) and 386 cases (qualita- Grade in introductory science courses,
college and universities (Sadler & tive). A mean value for such scores though other measures with similar
Tai, 200). The design of the current was substituted in these cases since correlations include: SAT Math,
survey included the use of student we did not want to eliminate students “other” science GPA, and last high
focus groups to comment on question from the dataset who had not taken school math grade. A central issue
formats, two pilot surveys that in- these exams. faced by those investigating the asso-
AP Exams and College Performance in Science
cluded 304 college science students, a Choice of controls ciation between AP exam performance
review by an advisory panel of college Our initial analysis compares the and college performance is that many
professmoros satn adp hpigrho pscrhiaotoel toepactihoenrs., Wem aenaanl yczoelldeg teh eg rpaedrecse nfotra gsetu odfe nmtsi ssinogth erre smpeoansusreess aonf da cfaoduemndic perfor-
and a stehpaarta, tfeo rre ltihabei lvitayr sitaubdlye ws hoefr ec oncweritnh, doinffleyre anbt oleuvte l1s- 2o%f h wighe rsec hmooisl singm faonrc ee aarceh a. lGsoi vpoesnit itvheilsy associated
3 cocllheagrea cchteemriissttircy, swtued ecnhtso stoeo tko usceo luirsste-wwoirske dine lbeitoiolong.y T, hceh etmwiost reyx cepwtiitohn cso lwleegree g trahdaets .no
the sursvteayn twdiacred, tiwzeo wteesetk ss caopraert a(SndA T oarn dA pChTy)s iwcs.a Ts heins tpearretidcu floarr a3n4a7ly csaiss es (quFaignutriet a3t iovfefe)r as na dg r3a8p6h ical repre-
where tchaesire rse s(pqounasleist wateirvee c)o.m Ap amreeda. n voanlulye s hfoowr ss uthceh p rsecdoicrteisv ew vaalsu es uofb AstPi tutesden itant itohne ussei ncga sthees rseisnuclets wpreo duced
Much cdaind b en dootn we toa nenth taon ceel iamcciunraactye stuedxeamnt ssc forroesm on t choel ldegaet acosuerts we ghraod ehsa, d nforto mta koeunr tshaemspel ee.x Camollse.g e science
and recall: careful design, contextual quantifying the magnitude of the dif- grades vary with AP exam score,
Choice of controls
cues, and participant relevance all play ference between students who take AP but they also vary with the other five
importOanut rro ilnesi.tial analysis compareso rt nhoet ,m aneda thno csoe wllehgo esc gorrea adt edsif ffeorre nstt udemnetass uwreist ho df iaffceardeenmti cl epvreelpsa ration.
In loafr ghei-gscha lsec hsourovle cyos,u rmseiswsinogr k inl ebvieolsl oogny th, ec hAePm exisamtr.y T ahnisd fi prsht sytseipc s. TAhsi se xppaerctteicdu, mlaorr ea nriagloyrosiuss levels of
responosnesl ya sreh onwots uthnec opmrmedoinc.t iIvne vadluesec roibf eAs tPh ee xdiaffmer esnccoerse tsh aotn n eceodl lteo ge choiguhr ssceh goroal dsceiesn, ce and mathematics
dealingq uwaitnh tmifyisisningg trheesp monasegsn, iltisutd- e obf et ehxep ldaiinffeedr beyn ccoem bpeettiwnge heynp ostthuedseesn ts wcohuros etsa mkeat cAhP w iothr hnigoht,e ra cnodll ege sci-
wise dethleotisoen iws thhoe ssicmoprlees at at nddi fmfeorste nt l(eDvoeulgsh oernty t,h Me eAlloPr e&x Jaiman. 2T0h06isa )f.irst steenpc ed gersacdreisb. Eeasc thh ger aph shows a clear
commodnilfyf earpepnliceeds o tphtiaotn .n Heeodw etvoe br, e explHaionweedv ebr,y a tcteommpptinegti tnog ac hcoyupnot ftohre sesm (oDnootuognhicearlltyy i,n Mcreealslionrg &as sociation
list-wisJiea nde 2le0t0io6na )a. s sumes that the the potential of taking an AP course between college grades and the six
data arHe mowisseinvge ra,t aratntedmomp. tSinhogu tldo aa ccorueqnut ifreosr tchoen tproolst efnotri adl eomf otgarkaipnhgic a n AhiPgh c osuchrosoel rmeqeuasiurreess of academic
large pceorcnetnrtoaglse ofof trh ed seammoplger eaxphihbiict andan adc aacdadeemmiicc pprreepparaartiaotnio dnif fderiefnfecrees nceasc ahmievoenmge nstt.u Idt sehnotusl da sb e noted that
missinwg reelslp. oFnasers f, rthoims a pbperionagch t mabauyl a raasma,o sntgu sdtuednentsts’ assk wilell la. Fnadr fkronmo bweilnegd ge tphrei aovre rtaog, eo grr audne raettlaaitneedd btyo ,s tudents
not bet athkei nmgo satn a AppPro cporiuatres eo psthioonu. ld btaeb iunlac lruasdae, sdtu. dTeanbtsl’e s k2i lsl hanodw kns otwhle- Peawrshoo ne acron rarne AlaPt isocnorse of 3 is equaled
We anableytzwede tehne paecracdenetmagiec o pf rmeipssa-ratioendg ve aprriioarb tole, os ra unndre lcaoteldle tog,e ta ckoinugr asne grabdy es.t u dents: earning a grade of “A” in
TTaabbllee 2 2: . CCoorrrreelalatiotino nC oCeoffiecfifeinctise notfs A ocfa dAecmadice Pmreicp aPrraetipoanr Catoivoanr iCatoevsariates
Regular
Last or
College SAT SAT Math Last Eng Science Honors AP
Grade Math Verbal Grade Grade GPA Grade Honors Calculus Score
College Grade 1.000
SAT Math 0.315 1.000
SAT Verbal 0.200 0.330 1.000
Last Math Grade 0.317 0.288 0.108 1.000
Last Eng Grade 0.219 0.134 0.200 0.343 1.000
Science GPA 0.306 0.254 0.190 0.387 0.378 1.000
Regular or Honors Grade 0.269 0.238 0.166 0.306 0.265 0.464 1.000
Honors 0.128 0.199 0.140 0.107 0.098 0.129 0.063 1.000
Calculus 0.244 0.415 0.203 0.192 0.159 0.247 0.196 0.281 1.000
AP Score 0.328 0.311 0.404 0.216 0.189 0.277 0.207 0.004 0.157 1.000
Among the nine academic performance measures, AP Score exhibits the highest
0 correlation with College Grade in introductory science courses, though other meSaciseuNcree es ducator
with similar correlations include: SAT Math, “other” science GPA, and last high school
math grade. A central issue faced by those investigating the association between AP
exam performance and college performance is that many other measures of academic
performance are also positively associated with college grades.
Figure 3 offers a graphical representation using the results produced from our sample.
College science grades vary with AP exam score, but they also vary with the other five
measures of academic preparation. As expected, more rigorous levels of high school
science and mathematics courses match with higher college science grades. Each graph
shows a clear monotonically increasing association between college grades and the six
high school measures of academic achievement. It should be noted that the average
grade attained by students who earn an AP score of 3 is equaled by students: earning a
grade of “A” in an honors or regular science course, scoring a 750 on their quantitative
Sadler and Tai Draft 8/13/07 — Do Not Duplicate or Quote Page 14