ERIC EJ1149722: The Validity of the Major Field Test in Psychology as a Programme Assessment Tool

2013·0.16 MB·English

by  ERIC

The validity of the Major Field Test in
Psychology as a programme
assessment tool
Shawn P. Gallagher & Shaun P. Cook
The Major Field Test in Psychology (MFT) is a standardised test designed to assess subject mastery at the
conclusion of an undergraduate career. Eighty-one graduating majors completed the MFT and 56 of them
also took a multiple-choice exam of questions drawn randomly from an introductory psychology test bank.
Like the MFT, the constructed exam was divided into four subscales. A second sample of 29 novice majors
also completed the MFT. For the advanced majors, total and subscale scores from the two tests were highly
correlated (all p<0.01). Advanced majors scored above national norms and were significantly better than the
novices. However, few courses could be linked to performance in the associated MFT subscales but general
academic indices like grade point average were highly predictive. The novice MFT scores were similar to the
norms for advanced majors. We conclude that the MFT measures broad, basic understanding, not advanced
mastery.
Keywords:Major Field Test; programme assessment; undergraduate psychology programmes.
L
ARGE RESEARCH INSTITUTIONS have conference attendees discussed psychology
long been compared internationally. education in the US and concluded, among
The Shanghai-based Academic Ranking other things, that periodic assessment could
of World Universities and the Times Higher not only improve the quality of undergrad-
Education World Universities Rankings are uate psychology programmes but also
influential but focus on quantifiable factors demonstrate the value of such programmes
like research funding and publications and, to accrediting bodies and benefactors
for the most part, ignore teaching (Altbach, (Halpern et al., 1993; McGovern, 1993). The
2012). This is also true for the rankings assessment movement has made significant
published by US News and World Report, the progress in the past two decades and, at the
most influential domestic scales of US insti- very least, has given faculty and administra-
tutions, which only assesses learning indi- tors a set of useful lenses through which to
rectly, using graduation rates (US News and review, evaluate, and improve their under-
World Report LP, 2013). However, in the face graduate programmes. Assessment experts
of the recent global recession, all US institu- have identified a number of domains that
tions, especially publicly funded ones, are warrant periodic review including Student
under increasing pressure to present objec- Learning, Curriculum, and Programme
tive measures of student learning. For better Resources (Dunn et al., 2007). Of these,
or worse, this kind of assessment, once the Student Learning is probably the most
domain of the individual schools, may even- important to demonstrate but the most diffi-
tually be used to compile national and inter- cult to quantify. Learning goals may not only
national rankings of institutions and their vary across institutions but also within insti-
major programmes. tutions as each psychology department
In 1991, the American Psychological attempts to prepare students for a wide
Association sponsored the St. Mary’s Confer- variety of career paths.
ence on Enhancing the Quality of Under- One company, Educational Testing
graduate Education in Psychology. The Services (ETS), produces a standardised
Psychology Teaching Review Vol. 19 No 2, Autumn 2013 59
© The British Psychological Society
Shawn P. Gallagher & Shaun P. Cook
assessment test, a Major Field Test (MFT), (see the Appendix). ETS identifies partici-
for psychology as well as several other under- pating institutions as those with five or more
graduate subject areas. These tests are students taking the test in the 2010–2012
‘designed to measure the critical knowledge period but there is no specific recruitment
and understanding obtained by students in a protocol. Some institutions administer it
major field of study (ETS, 2013)’. A report regularly to all graduating students (Stoloff
published by ETS not only suggests that their & Feeney, 2002) but the cost of $25 US per
products can be critical components of insti- exam might lead others, like ours, to admin-
tutional assessment but also calls for their ister the test only periodically and to only a
regular administration at all accredited US sample of students. The MFT test questions
institutions: cover a wide range of topics in psychology,
Post-secondary education today is not and test results are reported with a single
driven by hard evidence of its total score as well as subscores for Learning
effectiveness… A nationwide system of and Cognition (LRN), Perception, Sensory,
accountability needs to be developed Physiology, Comparative, and Ethology
within the context of efforts to monitor (PHYS), Clinical, Abnormal, and Personality
and improve higher education… We (CLIN), and Developmental and Social
recommend that… accrediting agencies (DEV). Although the objective results gener-
be charged with integrating a nationwide ated by a widely-used test give us a conven-
system of assessing student learning ient set of numbers to analyse and present, a
into… ongoing reviews of institutions of review of the literature prompted us to take
higher education (ETS, 2006). a closer look at the validity of the test
For US institutions, especially those who rely (Frazier & Edmonds, 2002; Stoloff & Feeney,
on government support and are annually 2002).
scrutinised by a tax-paying public, the Thousands of psychology majors from
message from ETS is clear: The MFT is not hundreds of institutions have taken the MFT,
only a useful tool for measuring ‘critical and many departments have used MFT test
knowledge and understanding’ but it is also results to evaluate their programmes and
an objective, quantifiable way to for a revise their curricula (Dolinsky & Kelley,
reviewing body, governmental or otherwise, 2010; ETS, 2013; Frazier & Edmonds, 2002;
to address accountability and, possibly, influ- Stoloff & Feeney, 2002). According to ETS
ence university rankings. The aim of this (2005), the MFT is a tool designed to ‘assess
study was to evaluate the validity of the mastery of concepts, principles, and knowl-
psychology MFT, a standardised multiple- edge expected of students at the conclusion
choice test, as a useful tool for internal or of an academic major in specific subject
external departmental assessment. Addition- areas’ (p.1). Presumably, performance on
ally, the strategies that we have employed can the test will improve as a student progresses
be applied to any such test that purports to through the undergraduate major and
assess advanced understanding in a disci- acquires a deeper and more comprehensive
pline. understanding of psychology. Advanced
Our psychology department serves over majors should outperform novices. However,
500 undergraduate majors and aspires to the reports from those using the MFT in
provide each one with a broad disciplinary programme assessment are, at best, incon-
foundation. One element of our curricular clusive about the factors that lead to high
review process includes the periodic admin- test scores. It is not surprising that MFT
istration of the MFT. More than 7000 scores consistently correlate highly and
students at 200 institutions have taken the significantly with general indicators of
140-question multiple-choice test since 2010 academic proficiency like grade point aver-
60 Psychology Teaching Review Vol. 19 No. 2, Autumn 2013
Validity of the Major Field Test
ages (GPAs) and SAT1 (College Board) Method
scores (Dolinsky & Kelley, 2010; Stoloff & Programme
Feeney, 2002), but some found that these Our undergraduate programme requires
measures have more predictive value than psychology majors to complete a minimum
discipline-specific indicators of acquired of 33 credit hours in psychology. Most
knowledge like the number of psychology students are between the ages of 18 and 22
courses taken (Dolinsky & Kelley, 2010; years and nearly half complete the under-
Stoloff & Feeney, 2002). In fact, success on graduate programme of 120 credits in four
the MFT has been linked to performance in years. Typical courses are worth three credit
only a few courses (Stoloff & Feeney, 2002) hours and hold three weekly classroom
or, as Frazier and Edmonds (2002) hours over a 14-week period. Lab courses
concluded, just one. Although some courses carry four credits and require additional
cover subjects named in the MFT subscale classroom hours that involve running exper-
titles, Stoloff and Feeney (2002) concluded iments or learning computer applications.
that students who take courses like All psychology students are required to take
Abnormal Psychology, Social Psychology, a three-credit course in General Psychology
and Developmental Psychology scored no and then two sequential four-credit courses
better on the respective subscales than those in Statistics and Experimental Design. They
who do not. must then choose at least one four-credit
The published literature challenges the advanced lab course in Child Development,
assertion that MFT scores reflect knowledge Cognitive Psychology, Learning and Motiva-
acquired at ‘the conclusion of an academic tion, or Sensation and Perception. Students
major’ in psychology. Although our informal select the remainder of courses according to
review of past MFT questions suggested that their interests and career objectives.
the psychology test had high face validity, it Percentage grades for each course are
also revealed that many of the questions converted to grade points that fall on a scale
dealt with foundational material covered in from 0 (65 per cent or lower) to 4.0 (95 per
our introductory psychology course (see the cent or higher). These points are then used
Appendix for sample questions). Results to calculate a grade point average for all
found in the published literature (e.g. courses (overall GPA) as well as psychology
Frazier & Edmonds, 2002; Stoloff & Feeney, courses in particular (major GPA). Psycho-
2002) also suggest that the MFT may be logy majors who fail to achieve a major GPA
assessing a student’s mastery of basic of 2.25 by the end of their second year are
concepts that are then reinforced in typically dismissed from the programme.
advanced courses. We, therefore, hypothe- Psychology majors who fail to achieve an
sised that MFT total scores and subscores overall GPA of 2.0 are not awarded degrees.
would correlate with results from an assess-
ment test that we constructed with questions Participants
randomly drawn from an introductory Eighty-one advanced psychology majors,
psychology textbook’s test bank. We also defined as students who were within one
hypothesised that, despite the fact that our month of completing the required
advanced majors historically perform at or minimum of 33 credits hours in psychology
above the national MFT means, a sample of (M=41.1 credits, SD=8.27), took the MFT as
novice psychology majors would also score well as other assessments and surveys.
close to the normative means. A second sample of novice majors (N=29),
1 The SAT is a standardised test used for admissions at most undergraduate institutions in the US. It contains
three major sections: Critical Reading, Mathematics, and Writing.
Psychology Teaching Review Vol. 19 No. 2, Autumn 2013 61
Shawn P. Gallagher & Shaun P. Cook
having completed no more than six credit from those of the MFT (χ2(4)=2.06, p>.05)
hours of psychology (two courses), also took which also includes questions (about 20 per
the MFT. We recruited the students two cent) that are not specific to any subgroup
weeks in advance and told them that they and used only in the total score calculation
were going to take a test of general subject (ETS, 2005).
knowledge. They were not instructed to
study for the tests and were given course Procedure
credit for participation, regardless of We administered tests with campus
performance. computers during single, uninterrupted
blocks of no more than two hours. Although
Materials proctors were available, no student
The assessment tests included the web-based requested assistance after receiving instruc-
version of the MFT and a 100-question intro- tions and starting a test. We counterbalanced
ductory psychology final exam (IPFE) that the order of the MFT and IPFE for the
we created with ExamView® software advanced majors at intervals of between two
(Version 5.2.0 FS Creations/now eInstruc- and 14 days. Students took the online MFT
tion, Denton TX) that accompanied an according to ETS protocol, and results were
introductory psychology textbook by James automatically submitted to and scored by
Kalat (2005). The questions are published in ETS. We then accessed overall and subscale
the book’s test bank (Meine & Kalat, 2005). scores through the ETS website. We could
This textbook was currently not in use at the not access answers to specific test questions
university and was chosen because of the or determine how many questions had been
author’s expertise in the development of answered. When students completed the
standardised tests including the Graduate IPFE, the software scored the tests and
Record Examination (GRE) in psychology, emailed results to the first author. We
also produced by ETS (Kalat & Matlin, entered the responses into a spreadsheet
2000). The ExamView® software automati- application that calculated subscale scores.
cally selected 100 multiple-choice questions
from a final exam question pool and Results
compiled them in the form of a single exam. Table 1 summarises mean GPA, SAT and
We then used the software to convert the MFT scores for the advanced and novice
exam to an HTML file that could be loaded majors. MFT scores are reported on a scale
onto a university server and administered via from 120 to 200; raw scores are not provided.
campus computers. The authors independ- For the advanced students, major GPAs
ently categorised each IPFE question into no (M=3.14, SD=0.57) were similar to their
more than two of the MFT subgroups, and overall GPAs (M=3.15, SD=0.52), and the
only questions nominated by both authors mean overall GPA did not differ significantly
were included in a given subscale score. from that of the previous year’s graduates
Distributions were as follows; 23 questions (z=1.35, p=0.18, two-tailed). MFT total and
were classified under LRN, 18 were classified subscale scores were significantly correlated
under PHYS, 21 were classified under CLIN, with all GPA and SAT measures (Table 2).
and 17 were classified under DEV. Four ques- The advanced majors (N=81) performed
tions were scored under two of the four well on the MFT (M=160.7, SD=15.0, Range:
subgroups, and the remaining 25 questions 126 to 196) and the total and subscale score
pertained to other areas, such as ethics and means were all above the corresponding
methodology, and were used only in the total normative means with the DEV subscale
score calculation. Surprisingly, the propor- mean being the only one not significantly so
tions of questions allotted to each subgroup (all other p<0.05, two-tailed z, N=7077).
in the IPFE were not significantly different Of the advanced majors who completed the
62 Psychology Teaching Review Vol. 19 No. 2, Autumn 2013
Validity of the Major Field Test
),
d
e
t
e
pl
m
o
edits c vice normal,e for all
ychology crN=7077). ormative–No – – – – z=1.32 z=1.28 z=–0.04 z=1.08 z=1.09 LIN=clinical, abde point averag
3 psple ( N ogy; CA=gra
omparison of GPA, SAT, and MFT total and subscale scores for Advanced Majors (>=3Novice Majors (<=6 psychology credits completed), and the Normative Sam AdvancedNovice NormativeAdvanced–NoviceMajorsMajorsSampleM (SD)M (SD)M (SD) GPA Overall3.15 (0.52)3.29 (0.59)–t(107)=–1.21 SAT Reading526.0 (79.5)555.6 (73.1)–t(78)=–1.41 SAT Math509.2 (89.0)535.0 (70.9)–t(78)=–1.13 SAT Writing522.0 (74.2)543.9 (81.3)–t(71)=–1.06 MFT Total160.7 (15.0)152.3 (12.9)156.1 (15.5)t(108)=2.69*, d=0.26 MFT: LRN59.1 (15.5)52.0 (12.8)55.6 (15.2)t(108)=2.22*, d=0.21 MFT: PHYS62.8 (15.5)56.2 (11.9)56.1 (14.7)t(108)=2.09*, d=0.20 MFT: CLIN60.9 (14.9)53.3 (14.8)56.4 (15.4)t(108)=2.37*, d=0.23 MFT: DEV57.9 (14.3)52.5 (15.3)55.5 (14.8)t(108)=1.71*, d=0.16 MFT=Major Field Test; LRN=learning and cognition; PHYS=perception, sensation, physiology, comparative, and etholand personality; DEV=developmental and social; Major GPA=grade point average for psychology courses; Overall GPcourses. *p<0.05, one-tailed.
C
1:
e
bl
a
T
Psychology Teaching Review Vol. 19 No. 2, Autumn 2013 63
Shawn P. Gallagher & Shaun P. Cook
s.
e
r
o
c
s
T
F
M
h
t
wi
s
e
al
c
am Scores and Academic S MFT: CLINMFT: DEV .77**.64** .71**.63** .63**.55** .69**.49** .62**.55** .67**.61** .64**.54** .30**.31** .50**.54** .30*.41** .51**.46** m an introductory text (Kalat, 2005)nsation, physiology, comparative,cial; Major GPA=grade pointajor Credits=Psychology Credits
ajors: Pearson correlations of Introductory Psychology Final Ex MFT: TotalMFT: LRNMFT: PHYS IPFE: Totalr(54).83**.75**.71** IPFE: LRNr(54).79**.74**.66** IPFE: PHYSr(54).71**.66**.68** IPFE: CLINr(54).72**.66**.61** IPFE: DEVr(54).68**.56**.63** Major GPAr(79).73**.70**.56** Overall GPAr(78).69**.65**.53** Major Creditsr(79).41**.43**.42** SAT Readingr(60).59**.52**.44** SAT Mathr(60).47**.44**.44** SAT Writingr(53).58**.50**.39** IPFE=Introductory Psychology Final Exam based on questions selected randomly frotest bank; MFT=Major Field Test; LRN=learning and cognition; PHYS=perception, seand ethology; CLIN=clinical, abnormal, and personality; DEV=developmental and soaverage for psychology courses; Overall GPA=grade point average for all courses. MCompleted.*p<0.05, **p<0.01, correlations for corresponding subscale scores are in bold.
M
d
e
c
n
a
v
d
A
2:
e
bl
a
T
64 Psychology Teaching Review Vol. 19 No. 2, Autumn 2013
Validity of the Major Field Test
y;
g
V o
MFT scores. MFT: DE .58** .71** .69** .55* ative, and etholverage for
Table 3: Novice Majors: Pearson correlations of Academic Scales with MFT: TotalMFT: LRNMFT: PHYSMFT: CLIN Overall GPAr(29).58**.59**.21.45* SAT Readingr(18).75**.70**.65**.62** SAT Mathr(18).77**.70**.79**.62** SAT Writingr(18).63**.70**.37.48* MFT=Major Field Test; LRN=learning and cognition; PHYS=perception, sensation, physiology, comparCLIN=clinical, abnormal, and personality; DEV=developmental and social; Overall GPA=grade point aall courses.*p<0.05, **p<0.01
Psychology Teaching Review Vol. 19 No. 2, Autumn 2013 65
Shawn P. Gallagher & Shaun P. Cook
MFT, 56 also completed the IPFE. Table 2 our advanced majors. This analysis did not
shows the Pearson correlations between the allow us to look at the effects of the three
elements of the MFT and IPFE. The MFT required courses taken by all advanced
total scores and subtest scores correlated students (General Psychology, Statistics and
significantly not only with their IPFE coun- Experimental Design I & II) and we
terparts but also with every other element of excluded broad content courses that could
the IPFE. The IPFE LRN and PHYS subscale be applied to all four subscales (History and
scores correlated best with their MFT coun- Systems, Tests and Measurements). Using
terparts. Major GPA, overall GPA, and major t-tests to compare the mean scores of those
credits completed also correlated signifi- who did and did not take specific courses, we
cantly with all MFT scores (all p<0.05). found no content-specific effects for 17
All the novice majors (N=29), had courses (all p>0.10) which, based on course
completed a course in introductory descriptions, should contribute to specific
psychology, and 11 of the 29 had also MFT subscale scores (Table 4). The litera-
completed a course in child development. ture provides little evidence linking specific
For the novice majors, MFT total and courses to subscale scores but Dolinsky and
subscale means were all significantly lower Kelley (2010) noted dramatic improvements
than those of the advanced majors in PHYS subscale scores after the implemen-
(M=152.3, SD=12.9, Range: 131 to 174), tation of a required second-year course in
although effect sizes were modest (Cohen’s physiological psychology. In our case, those
d). Novice means were not significantly who took physiological psychology (N=29),
lower than the normative means provided by outperformed their peers in all MFT meas-
ETS (all p>.05, one-tailed z). Five novice ures (all p<0.01, one-tailed t). Those taking
majors had MFT total scores that placed physiological psychology also had higher
them above the 80th percentile for the indi- major GPAs (t(79)=2.34, p=0.01, one-tailed)
vidual national normative scores (ETS, and completed more credits (t(79)=5.76,
2013). MFT total scores and most subscale p<0.01, one-tailed) than their counterparts.
scores were highly correlated with GPA and A similar broad, nonspecific effect was
SAT scores (Table 2). observed for History and Systems. Both of
We performed a stepwise linear regres- these courses tend to attract students with
sion analysis to predict MFT total scores graduate school aspirations.
from Major GPA, Major Credits, SAT In a final, exaggerated effort to find a
Reading, SAT Math, and SAT Writing scores. link between course content and an MFT
Overall GPA was highly collinear with Major subscale score, we decided to compare two
GPA (r=.92) and, therefore, excluded from specific groups of advanced majors. Students
the analysis. Each of the predictor variables who take physiological psychology often
had a significant (p<.05) zero-order correla- have plans to pursue graduate training in
tion with MFT total score (Table 2), but only experimental psychology and we were not
Major GPA (β=.44), Major Credits (β=.28), surprised by the performance of this highly
and SAT Writing (β=.29) had significant motivated group. We compared these
partial effects in the full model (all p<.01). students to a second group of motivated
The model employing only these three students with postgraduate plans in clinical,
predictors accounted for 62 per cent of the rather than experimental, psychology. In our
variance in MFT total score, (F(3,51)=27.96, case, such students can be found by identi-
p<.001). fying those who have taken both abnormal
Like Stoloff and Feeney (2002), we exam- psychology and personality theory, two topics
ined the effect of completing specific that fall firmly within CLIN subscale of the
courses on MFT performance for all courses MFT. We compared students who took both
completed by 10 per cent to 90 per cent of of these courses but not physiological
66 Psychology Teaching Review Vol. 19 No. 2, Autumn 2013
Validity of the Major Field Test
Table 4: Courses taken by 10 to 90 per cent of advanced majors for which no
content-specific effect on MFT was observed. Each course is listed along with the
abbreviation of the subscales for which a specific effect might be anticipated.
Required courses taken by all students (General Psychology, Statistics and Experimental
Design I & II) and those with broad content (History and Systems of Psychology,
Tests and Measurements) were not included.
Abnormal Psychology –CLIN
Behavior Modification – LRN, CLIN
Childhood Disorders – CLIN, DEV
Cognitive Psychology – LRN, PHYS
Counseling Strategies – CLIN
Child Development Lab - DEV
Drug Addiction – CLIN, PHYS
Family Systems – CLIN
Health Psychology – PHYS
Human Adjustment – CLIN
Human Relations – CLIN, DEV
Industrial/Organisational Psychology –
DEV
Learning and Motivation – LRN, PHYS
Lifespan and Human Development – DEV
Personality Theory – CLIN
Sensation and Perception – PHYS
Social Psychology – DEV
LRN=learning and cognition; PHYS=perception,
sensation, physiology, comparative, and ethology;
CLIN=clinical, abnormal, and personality;
DEV=developmental and social.
psychology (N=37, M=63.27, SD=13.4) to samples, we suspect that the seemingly para-
those who took physiological psychology but doxical result is due to exposure to broad,
not abnormal psychology or personality basic content.
theory (N=16, M=60.81, SD=19.9). Contrary
to our hypothesis, those who took physio- Discussion
logical psychology but not the two clinical Our department has been using the MFT for
courses had a higher mean CLIN subscale several years as part of our periodic review
score (M=66.7, SD=11.01) than those who process. Although the results are objective
took the clinical courses but not physiolog- and simple to compile and present,
ical psychology (M=58.5, SD=15.3). These published reports compelled us to evaluate
two groups did not differ significantly in the test’s validity (Frazier & Edmonds, 2002;
mean major GPA (t(51)=1.16, p=.25, two- Stoloff & Feeney, 2002). Previous studies
tailed), overall GPA (t(51)=.836, p=.41, two- have challenged the notion that multiple
tailed), or SAT scores (all p>0.1, two-tailed t). choice exams can measure critical thinking
The physiological psychology group did, skills (Stanger-Hall, 2012) and found that
however, complete more major credits they typically over-estimate general content
(t(51)=5.31, p<0.01). For these matched mastery (Funk & Dickson, 2011). The results
Psychology Teaching Review Vol. 19 No. 2, Autumn 2013 67
Shawn P. Gallagher & Shaun P. Cook
of our investigation are consistent with these supplemental psychology education or
findings and have led us to four conclusions. training; two had completed only General
First, MFT total and subscale scores can be Psychology, and three had completed
predicted by performance on a comprehen- General Psychology and Child Development.
sive introductory psychology exam. Second, We suspect that exceptional novice students
although our advanced students performed from any university would do well on the
well on the MFT, as predicted by Stoloff and MFT, and the gaps we observed between
Feeney (2002), we found no link between advanced and novice scores might be even
specific courses and MFT subscale scores. smaller for departments with two-semester
Third, novice majors can perform well on introductory psychology requirements.
the MFT despite having completed no more Although they will not have data that they
than two psychology courses. Finally, can directly compare to a normative sample,
performance on the MFT appears to be a we recommend that, if a department aspires
function of overall academic potential as to measure the broad, basic content knowl-
measured by standardised tests and GPA. edge of its advanced majors, they construct
In our opinion, the sample questions an IPFE of their own with questions from a
provided by ETS (Appendix A) and the ques- comprehensive introductory psychology
tions that we have seen on the actual tests textbook.
assess basic, introductory-level knowledge. Although our specific claim that the MFT
Although we observed significantly different fails to validly assess mastery might be new,
MFT scores between novice and advanced our results are consistent with many previous
students, the question of what led to these reports. Like Stoloff and Feeney (2002), we
differences remains. We suspect that these found that although psychology credits
differences are due to the fact that the completed was significantly correlated with
advanced students have had more opportu- MFT total, GPA, either Overall or Major,
nities to review and discuss basic material accounted for approximately 50 per cent of
common in introductory courses. The the variance in MFT total while credits
students who do well on the MFT might well completed accounted for less than 20 per
have a deep understanding of psychology, cent of the variance. In another study
but they also have had many opportunities to exploring MFT correlates, Frazier and
review core content; basic Freudian princi- Edmonds (2002) found that, of all the
ples get reinforced through courses in academic indicators evaluated, MFT scores
Personality Theory or Abnormal Psychology were predicted only by performance in a
and basic Neuroanatomy is reviewed in single in-depth, and closely mentored,
courses like Sensation and Perception. Our research methods course. The authors noted
top-scoring novices are exceptional students that those who excel in this class are typically
who have performed well in all of their those planning on graduate school and, we
college courses and have probably mastered suggest, that they were the ones who made a
the material they have encountered so far. point of mastering the basics from the begin-
However, regardless of how exceptional a ning.
novice student happens to be, a test In a four-year longitudinal study,
‘designed to assess mastery of concepts, prin- Dolinsky and Kelly (2010) noted impressive
ciples, and knowledge expected of students gains in MFT scores after converting
at the conclusion of an academic major’ Abnormal Psychology, Physiological
(ETS, 2005) should contain material well Psychology, and History and Systems from
beyond his or her reach. Although one of optional to required courses. Although it is
our five top-scoring novice students took an likely that the course in Abnormal
advanced placement psychology course in Psychology contributed to the gains in the
high school, none of the others had any CLIN subscale scores and that Physiological
68 Psychology Teaching Review Vol. 19 No. 2, Autumn 2013