ERIC EJ899304: Readiness Assessment Tests versus Frequent Quizzes: Student Preferences

2009·0.2 MB·English

by  ERIC

International Journal of Teaching and Learning in Higher Education 2009, Volume 21, Number 2, 181-186

http://www.isetl.org/ijtlhe/ ISSN 1812-9129

Readiness Assessment Tests versus Frequent Quizzes:
Student Preferences

Suzanne E. Weinstein Shao-Wei Wu
The Pennsylvania State University York College, CUNY

This study compares the effectiveness of two different assessment techniques; readiness assessment
tests (RATs) and frequent quizzing. We report student perceptions of the impact of these techniques
on the number of readings done prior to the class period, thorough reading of assignments, ability to
follow class discussions, ability to participate in class, ability to prepare for exams and exam scores.
We also examined student’s overall preferences for assessment technique as well as how preferences
varied by learning styles. Readiness assessment tests were generally better than frequent quizzes at
encouraging students to do the readings prior to class, follow class discussions, and participate in
class. A majority of students preferred readiness assessment tests to frequent quizzes. However,
whereas global and/or intuitive learners preferred the readiness assessment tests, sequential and/or
sensing learners preferred the quizzes.

Frequent assessment enhances student learning. shown that the JiTT technique is associated with
The more opportunities students have to work actively increased number of students who do the readings
with course material and receive feedback, the better (Howard, 2004), student perceptions of improvement in
the chances that they will learn it. Classroom critical thinking ability (Cookman, 2004), and enhanced
assessment techniques (Angelo & Cross, 1993), exam scores (Benedict & Anderton, 2004).
frequent quizzing (Maki & Maki, 2001; Roediger & Frequent quizzing also helps students to keep up
Karpicke, 2006), and readiness assessment tests with the material and reduces the importance of each
(Carkenord, D.M., 2004; Padilla-Walker, 2006) are single test, which can mitigate students’ perceived need
some of the many available assessment strategies. to cheat. Instructors can use information gathered from
Given so many possible strategies, how does an quiz performance to help students prepare for exams.
instructor make a choice? The most important factor in Instructors who use frequent quizzing typically employ
choosing a strategy is the match between the strategy multiple choice questions (Maki & Maki, 2001;
and the learning objective. Beyond that, student Marcell, 2005). Research has shown that frequent
perceptions and strategy effectiveness are important quizzing, when compared to a few long tests, increases
considerations. the chances that students will do the readings and is
Readiness assessment tests (RATs) require students preferred to fewer tests by students who have
to respond to questions about the assigned readings experienced it (Connor-Greene, 2000).
prior to class discussion (Cookman, 2004; Howard, Studies of assessment effectiveness, such as those
2004; Marrs Blake & Gavrin, 2003). The RATs can be cited above, typically report an overall preference for
done on paper at the beginning of class or electronically the assessment type or average increase in the
before coming to class. Theoretically, any question type measured outcome (e.g. student performance).
could be used to assess students’ readiness to engage in However, the efficacy of any assessment strategy for
discussion, but most instructors using this technique an individual student may depend on how well it
employ either open-ended questions, such as short matches the student’s learning style. According to
answer or essay (Corkenord, 2004; Connor-Greene, Cassidy (2004) “there is general acceptance that the
2000; Cookman, 2004; Marrs, Blake & Gavrin, 2003), manner in which individuals choose to or are inclined
or a combination of both open-ended and multiple to approach a learning situation has an impact on
choice questions (Benedict & Anderson, 2004; Howard, performance and achievement of learning outcomes.”
2004). The major objectives of RATs are to encourage For example, Zywno and Waalen (2002) showed that
students to come to class prepared for discussion and to engineering instruction enhanced by hypertext and
keep up with the material to prevent cramming for an multimedia was more effective than traditional
exam. When the instructor adapts her behavior based on instruction for Active and Global learners but less
responses to readiness assessments, she is doing “Just- effective for Verbal learners. These individual
in-Time” Teaching (JiTT) (Benedict & Anderson, differences impact learning through preferences for
2004; Howard, 2004; Novak et al. 1999; Watson & the type of information, the sensory channel through
Temkin, 2000). This strategy allows her to spend more which information is perceived, the way information
time on certain concepts if student responses indicate is organized, the way it is processed and the way that
the need and to incorporate student thoughts and individuals come to understand (Felder & Silverman,
examples into the class discussion. Researchers have 1988). Thus, learning styles may impact the
Weinstein and Wu Readiness Assessment Tests versus Frequent Quizzes 182

effectiveness of any assessment strategy for any single Student perception surveys. On each of four
student. surveys designed for this project, students rated four
The aim of the present study was to compare RATs statements about the RATs or quizzes on a five-point
and frequent quizzing with respect to the impact of each Likert-scale from strongly disagree to strongly agree.
strategy on the number of readings completed, students’ The statements addressed the effect of the assessment
thorough reading of assignments, and students’ ability strategy on thorough reading of the material, ability to
to follow class discussions, participate in class, and follow class discussions, ability to participate in class,
prepare for exams. We also assessed the ability of each and ability to prepare for the exam. On the fourth (final)
strategy to predict exam scores. A secondary goal was survey students indicated their preference for RATs or
to determine the impact of learning styles on student quizzes. They were also asked to provide any
preference for assessment strategy. comments on how the assessment methods helped their
learning as well as suggestions they had for improving
Method the two assessment methods. The links to these surveys
were sent to students within a week after each unit
Participants exam. Students completed the surveys online and the
results were returned electronically to the second
Participants were 51 college students (29 women, author.
22 men) in the first author’s upper level psychology Index of Learning Styles. This is a 44-item forced
course, “The Psychology of Fear and Stress,” during choice, Myers-Briggs Type Inventory-like questionnaire
the spring semester of 2006 (final enrollment = 60). (Felder & Soloman, 1991). It combines aspects of several
The class met twice per week. Thirty-six students (22 learning style models, including Kolb (1984) and Jung-
women) completed all four surveys plus the learning Myers-Briggs (Felder and Silverman, 1988). The test-
styles questionnaire. Fifteen students didn’t respond to retest reliability of the ILS for a four-week interval
one or more of the surveys or the learning styles ranges from .73 to .87 depending on the learning style.
questionnaire. These students were excluded from the The instrument is administered online and easily
data analysis. understood by students (Zywno, 2003). The responses
indicate where individuals fall along four learning styles
Materials dimensions, active-reflective (doing something with the
information vs. thinking about it), sensing-intuitive
Required course assignments. The four-unit (obtaining data through senses vs. indirect perception),
course included two different assessment strategies. visual-verbal (preference for pictures, graphs, charts, etc.
During the first and third units students completed vs. verbal information, either written or spoken) and
readiness assessment tests (RATs) online prior to sequential-global (learning in a step-by-step fashion vs.
class. The RATs consisted of two to three open-ended holistically).
questions asking for students to describe the major
point of the article or areas that were most interesting Procedure
and/or least understood. These broad questions were
used to prevent students from skimming through the Participants completed the necessary course
readings in search of answers to detailed questions. assignments (see above), the Index of Learning Styles,
Each RAT was worth four points and students were and four brief surveys, one after each unit exam. All
required to complete five of six, for a total of 20 students in the class were offered extra credit (up to 1%
points. of the total grade if they completed the learning styles
During the second and fourth units, students instrument plus all four surveys) to participate in the
completed short online quizzes, which were completed study. In addition, an alternative extra credit assignment
by midnight on Fridays. Quizzes included 10 multiple was offered for students who did not want to participate
choice questions. Students were required to complete in this study. The second author, who was not an
all four quizzes, each worth five points, for a total of instructor, obtained informed consent from interested
20 points. See Table 1. The quizzes were students during a class period early in the semester and
administered at the end of a week and covered collected data to prevent the instructor from knowing
material from two class periods, whereas the RATs which students participated until the course was
were administered prior to each class period. Thus, completed. The informed consent assured students that
there were fewer quizzes than RATs. their responses to the study instruments were
Students took an in-class, 50-point exam at the anonymous; the instructor would not know who had or
end of each unit. The exams included both multiple had not agreed to participate in the study until after the
choice and essay questions. course.
Weinstein and Wu Readiness Assessment Tests versus Frequent Quizzes 182

Table 1
Assessment Timeline
Unit 1 Unit 2 Unit 3 Unit 4
Definitions of Stress Health Effects Stress & Depression, Anxiety and Stress
Physiology of of Stress Moderators of Disorders, Cognitive
Stress Response Stress, Coping Aspects of Anxiety
Date 1/12 1/17 1/19 1/26 2/3 2/10 2/16 2/21 2/23 3/14 3/16 2/24 3/31 4/4
Assessment RAT1 RAT2 RAT3 Ex 1 Q1 Q2 Ex 2 RAT4 RAT5 RAT6 Ex 3 Q3 Q4 Ex 4
Points 4 pts. 4 pts. 4 pts. 50 pts. 5 pts. 5 pts. 50 pts. 4 pts. 4 pts. 4 pts. 50 pts 5 pts. 5 pts. 50 pts.

Data Analysis 149, r = .48, p < .008). In addition, students rated the
second RAT significantly higher than both quizzes (T =
Kolmogorov-Smirnov tests revealed the student 231, r = .49, p < .008; T = 203, r = .52, p < .008) in
perception data were not normally distributed. Thus, enhancing their ability to participate in class. No
Friedman’s ANOVAs were performed to test significant differences were found with respect to
differences in student perceptions of the effect of RATs encouraging students to read articles more thoroughly
vs. frequent quizzing on reading the articles more or enhancing students' ability to prepare for exams. See
thoroughly, following class discussions, participating in Figure 1.
class, and preparing for exams. Friedman’s ANOVA is
a non-parametric technique used to test “differences RATs vs. Frequent Quizzing – Readings
between experimental conditions when there are more
than two conditions and the same participants have The rankings of the percentage of readings
been used in all conditions” (Field, 2005). The same completed prior to class were significantly different (χ2
statistic was used to investigate the effect of assessment (3) = 70.55, p < .05) across the four units. The
method on the number of assigned readings completed Wilcoxon test suggested that the number of readings
prior to the class period and exam scores. that students completed prior to class was significantly
Learning styles were calculated by adding one higher during the units that required RATs (mean =
point for each response that endorsed a particular 3.36) than during the units that required quizzes (mean
dimension (11 questions for each dimension). Scores of = 1.65).
1 – 3 are considered fairly well-balanced; 5 - 7 indicates
a moderate and 9 - 11 a strong preference (Felder & RATs vs. Frequent Quizzing - Exam scores
Soloman, 1991). Because scores of 1 – 3 indicate a
person without a strong preference for one learning The four exam score averages were significantly
dimension over the other, only students with scores of 5 different (χ2 (3) = 11.30, p = .01). The Wilcoxon test
or greater were included in these analyses. A Chi- suggested that the first exam score average was
square test was performed on the learning styles significantly higher than the second exam score average
dimensions to explore the relationship between learning (T =519, r = .49). See Figure 2.
styles and preference for assessment method.
Student Preference
Results
About 56% of the students reported a preference
RATs vs. Frequent Quizzing – Student Perceptions for RATs and 33% reported a preference for quizzes.
The remainder of students reported that their
Student perceptions differed significantly with preference for one strategy over the other depends on
respect to enhancing their ability to follow class the content. Students who preferred RATs indicated in
discussions (χ2 (3) = 15.65, p < .001) and to participate their open-ended responses that the questions helped
in class (χ2 (3) = 13.17, p < .01). The post hoc them look at the overall meaning of the articles and
Wilcoxon signed-rank test with a Bonferroni correction focus on the main points. In addition, having the
set at .0083 was used to further explore the differences. RATs due before class helped them prepare to
This procedure is used to compare two dependent participate in the classroom discussions. Students who
conditions when the data are nonparametric. The preferred frequent quizzes reported that their
Wilcoxon test suggested that the students rated the first preference was due to quiz questions showing them
RAT significantly higher than the first quiz in what to expect from exams and having only one
enhancing their ability to follow class discussions (T = correct answer.
Weinstein and Wu Readiness Assessment Tests versus Frequent Quizzes 183

Figure 1
Student Perceptions on the Effectiveness of RATs and Frequent
Quizzes on their Ability to Follow Class Discussion and Participant in Class

Figure 2
Exam Scores
Weinstein and Wu Readiness Assessment Tests versus Frequent Quizzes 184

RATs vs. Quizzes - Student Learning Styles prepared but still need external motivation (such as a
RAT) to encourage them to do so.
There was a significant association between the Although students’ subjective reports revealed that
Sequential-Global dimension and preference for RATs RATs and frequent quizzing had equal effects on their
or frequent quizzes, χ2 (1) = 7.00, p < .05. The strength ability to prepare for exams, the actual exam scores
of the relationship was significant (Cramer’s V = .73, p were different for the first unit (RAT) than for the
< .01). The result suggests that individuals classified as second unit (frequent quizzing). Lower scores for the
sequential are more likely to prefer quizzes over RATs, second exam is typical in this course due to the mixed
while individuals classified as global are more likely to course content (“psychology” and “biology”). The
prefer RATs over frequent quizzes (see Table 2). second unit covers the physiological effects of stress on
The association between the Sensing-Intuitive the major body systems, material with which
dimension and preference for RATs or frequent quizzes psychology majors, who make up the vast majority of
was marginally significant (χ2 (1) = 4.11, p = .058). this course, typically have less experience. Thus, it
The strength of the relationship was significant seems reasonable for exam scores to be lower for this
(Cramer’s V = .45, p < .05). The result suggests that unit than the other 3 units, which contain less biology.
individuals categorized as sensing are more likely to Although the data indicate that, overall, the RATs
prefer frequent quizzes over RATs and individuals were more helpful to students than the frequent quizzes,
categorized as intuitive are more likely to prefer RATs they also suggest that student learning styles had an
over frequent quizzes (see Table 3). No significant impact on the types of assessment methods students
association was found between Active-Reflective or preferred.
Visual-Verbal dimensions and assessment preferences. The preference for the open-ended RATs by
students with a tendency for intuitive and/or global
Table 2 learning aligns well with the definition of these learning
Preference for Frequent Quizzes styles (see Soloman & Felder, 1991). Intuitive learners
or RATs by Learning Style – Sequential or Global prefer seeing relationships over learning facts and are
Learning Style more comfortable with abstract concepts than sensing
learners. Global learners are able to make connections
Sequential Global Total
in content without the need for step-by-step
RATs 2 5 07
explanations. Thus, it makes sense that individuals who
Frequent Quizzes 6 0 06 prefer either of these styles would prefer questions that
Totals 8 5 13 require them to comment on the readings overall by
stating the main points or the areas about which they
Table 3 still have questions. For example, they might be asked
Preference for Frequent Quizzes to explain the main point of a chapter that addresses
or RATs by Learning Style – Sensing or Intuitive why we have a stress response. In contrast, sensing
Learning Style and/or sequential learners may have been more likely to
Sensing Intuition Total prefer the multiple choice quizzes due to their comfort
learning facts in a linear, step-by-step fashion. The
RATs 05 6 11
multiple choice questions were more likely to address
Frequent Quizzes 08 1 09
specific facts, such as the hormones involved in the
Totals 13 7 20
stress response, and sequential events, such as the
cascade of physiological events that make up a stress
Discussion response.
Constraints related to the practical aspects of the
The results of the current investigation suggest that course, such as the timing and question-type differences
both readiness assessment tests and frequent quizzing between RATs and frequent quizzing, and the small
are equally effective at encouraging students to read sample size suggest caution in interpreting these data.
articles thoroughly and prepare for exams. However, The RATs were due prior to a single class period and
the RATs generally enhanced students’ ability to follow addressed a single reading. The quizzes, however,
and participate in class discussion more than the occurred at the end of a week after two class periods
frequent quizzes did. These results are most likely a and typically addressed two readings. Perhaps students
result of the fact that students completed more of the prefer to do their course work during the week rather
readings before class for RATs than they did for than worrying about taking a quiz by Friday night.
quizzes. Student narrative responses to the open-ended Also, the RATs tended to be subjective and were scored
questions in the student perception survey suggest that based on whether or not students completed the
students recognize the value of coming to class assignment rather than correctness of the responses. In
Weinstein and Wu Readiness Assessment Tests versus Frequent Quizzes 185

contrast, the quiz scores were more objective – answers time teaching in a history of photography course.
were either right or wrong. Thus, the scoring variation Proceedings of the International Society for the
may be an explanation for the students’ RAT Scholarship of Teaching and Learning: The
preference. In addition, although the study design Scholarship of Teaching and Learning:
attempted to balance the assessment methods across Perspectives, Intersections, and Directions.
more biologically-oriented and more psychologically- Bloomington, IL.
oriented topics, this could not be done perfectly. It Howard, J. R. (2004). Just-in-time teaching in sociology
might be that the students preferred the topics or how I convinced my students to actually read the
associated with the RATs over those associated with the assignment. Teaching Sociology, 32, 385-390.
quizzes. Alternatively, material for which quizzes were Kolb, D. A. (1984). Experiential learning. Englewood
used might be more difficult than that for which RATs Cliffs, NJ: Prentice Hall, Inc.
were used. Future studies that address these Felder, R. M., & Silverman, L. K. (1988). Learning and
methodological issues are warranted. teaching styles in engineering education.
Further investigation is also important to Engineering Education, 78, 674-681.
substantiate our interpretation of the learning styles Felder, R. M., & Soloman, B. A. (1991). Index of
data. If the preference for RATs vs. quizzes is a result Learning Styles (ILS). Retrieved from
of variation in question format rather than other aspects http://www4.ncsu.edu/unity/lockers/users/f/felder/pu
of the assessment method, a simple follow-up blic/ILSpage.html
investigation in which only question type is varied Field, A. (2005). Discovering Statistics Using SPSS,
could substantiate the conclusion. In addition, future Thousand Oaks, CA: SAGE Publications.
studies might also explore how self-regulated learning, Maki, W. S. & Maki, R. (2001). Mastery quizzes on the
which is a person factor, is related to preference for web: Results from a web-based introductory
assessment method (see Pintrich, 2004; Zimmerman, psychology course. Behavior Research Methods,
1998). Instruments & Computers, 33, 212-216.
Nevertheless, the present data do suggest some Marrs, K. A., Blake, R. E., & Gavrin, A. D. (2003).
reasonable conclusions. Firstly, if an instructor’s Web-based warm-up exercises in just-in-time
objective is for students to do the readings prior to class teaching: Determining students’ prior knowledge
and be prepared to participate fully in class discussions, and misconception in biology, chemistry, and
she should consider using RATs to provide some physics. Journal of College Science Teaching, 33,
external motivation. However, if an instructor’s 42-47.
objective is for students to learn the material in any way Marcell, M. (August, 2005). Effectiveness of online
possible and/or there isn’t enough time to score student quizzing in increasing class preparation and
responses every class period, he might consider weekly participation. Poster presented at the annual meeting
quizzes as an alternative. In either case, a good strategy of the American Psychological Association,
for addressing the variation in learning styles is to Washington, D.C.
include open-ended, subjective questions and objective, McKeachie, W. J., & Svinicki, M. (2006). McKeachie’s
multiple choice questions in a course assessment teaching tips: Strategies, research and theory for
strategy. college and university teachers. New York;
Houghton Mifflin Company.
References Novak, G. M., Patterson, E. T., Gavrin, A.D., &
Christian, W. (1999). Just-in-time teaching:
Benedict, J. O., & Anderton, J.B. (2004). Applying the Blending active learning with web technology.
just-in-time teaching approach to teaching Upper Saddle River, NJ: Prentice Hall.
statistics. Teaching of Psychology, 31, 197-199. Padilla-Walker, L. M. (2006). The impact of daily extra
Carkenord, D. M. (1994). Motivating students to read credit quizzes on exam performance. Teaching of
journal articles. Teaching of Psychology, 21, 162- Psychology, 33, 236-239.
164. Pintrich, P. R. (2004). A conceptual framework for
Cassidy, S. (2004). Learning styles: An overview of assessing motivation and self-regulated learning in
theories, models, and measures. Educational college students. Educational Psychology Review,
Psychology, 24, 419-444. 16, 385-407.
Connor-Greene, P. A. (2000). Assessing and promoting Roediger, H. L., & Karpicke, J. D. (2006). The power of
student learning: Blurring the link between testing memory: basic research and implications for
teaching and testing. Teaching of Psychology, 27, educational practice. Perspectives on Psychological
84-88. Science, 1, 181-210.
Cookman, C. (2004). Improving students’ critical Watson, C., & Temkin, S. (2000). Just-in-time
thinking skills through Internet technology: Just in teaching: balancing the competing demands of
Weinstein and Wu Readiness Assessment Tests versus Frequent Quizzes 186

corporate America and academe in the delivery of has taught courses in introductory psychology,
management education. Journal of Management physiological psychology, health psychology, and fear
Education, 24, 763-778. and stress. She has worked in faculty development, with
Zimmerman, B. J. (1998). Academic studying and the a focus on assessment, since 2002. She is interested in
development of personal skill: A self-regulatory how individual differences influence student
perspective. Educational Psychology, 33, 73-86. perceptions and performance in both face-to-face,
Zywno, M. S., & Waalen, J. K. (2002). The effect of online and hybrid learning environments. She is
individual learning styles on student outcomes in currently providing support for the university-wide
technology-enabled education. Global Journal of efforts to improve program assessment processes in all
Engineering Education, 6, 35-44. departments across the multi-campus university.
Zywno, M. S. (2003). A contribution to validation of
score meaning for Felder-Soloman’s Index of SHAO-WEI WU, Ph.D., is the Coordinator of
Learning Styles. Proceedings of the American Assessment at York College, CUNY. She is also co-
Society for Engineering Education Annual chair of the college-wide Outcomes Assessment
Conference & Exposition: Staying In Tune With Committee. She received her Ph.D. in Instructional
Engineering Education. Nashville, TN. Systems from Penn State University. Her research
interests are in the area of instructional design and
____________________________ individual differences. She is also involved in several
institutional research and assessment projects.
SUZANNE WEISTEIN, Ph.D., is the Director of
Instructional Consulting, Assessment and Research at Acknowledgements
the Schreyer Institute for Teaching Excellence at the
Pennsylvania State University. She received her Ph.D. We would like to thank the reviewers for their
in psychology in 1999 from Penn State and holds a thoughtful comments, which helped us to strengthen the
courtesy appointment in that department. Her area of manuscript.
expertise involves the mind/body connection and she