ERIC EJ1058490: Business Education Innovation: How Common Exams Can Improve University Teaching

2010·0.14 MB·English

by  ERIC

American Journal of Business Education – September 2010 Volume 3, Number 9
Business Education Innovation:
How Common Exams
Can Improve University Teaching
Darian Unger, Howard University, USA

ABSTRACT

Although there is significant research on improving college-level teaching practices, most
literature in the field assumes an incentive for improvement. The research presented in this paper
addresses the issue of poor incentives for improving university-level teaching. Specifically, it
proposes instructor-designed common examinations as an incentive for teaching improvement and
uses empirical data from business school student tests to illustrate the utility of such assessments.
Results were drawn from almost 250 college students who had different professors for the same
course. Comparing the data from a common assessment revealed important differences about
what students learn and created opportunities and incentives to improve teaching practices in a
way few other methods can.

Keywords: Business education, assessment, pedagogy improvement

INTRODUCTION

T eaching and pedagogy are critical to college and university education. It is particularly important in
business education because of the combination of quantitative and qualitative skills that students
must master. Considerable research suggests that effective and skilled instruction is the prime
variable in student achievement (Harris and Sass, 2008; Hill et al., 2005; Jepsen, 2005; Nye et al. 2004; Marzano et
al., 2001). To achieve this level of instruction, prior research explained below proposes the importance of
accountability, but most literature does not focus on post-secondary education, where incentive systems vary
significantly by university or college, and where accountability is usually left to an internal reviewing entity or an
accreditation board.

Recent national efforts to improve primary and secondary level education have focused on improving
pedagogy and creating common learning goals. These methods, including the No Child Left Behind Act and the
“Race to the Top” inter-state competition for federal education funds, emphasize competitive benchmarking, a
practice common to industry but less widespread in education (Cohen-Vogel and McLendon, 2009; Reback, 2008;
Smith, 2008).

In the post-secondary education market, such standardization is considerably less enforceable (Benjamin,
2008). Although accrediting agencies do rate some college programs, most universities self-regulate. In
combination with tenure systems, this can lead to potentially reduced incentives for teaching improvement.

The teaching incentive problem is a well-studied field in primary and secondary education (Figlio and
Kenny, 2007; Kane and Staiger, 2002; Lavy, 2002). However, in college and university settings, research on
pedagogy has generally assumed that incentives for improvement exist (Abraham and Karns, 2009). This paper
acknowledges the incentive problem and proposes common examinations as a solution. Data for this empirical
research uses results from 245 college students who had different professors for the same course. Comparing the
data from a common assessment revealed important differences about what students learn. The comparisons created
both opportunities and incentives to improve teaching practices.
67
American Journal of Business Education – September 2010 Volume 3, Number 9
LITERATURE REVIEW AND PROBLEM

This section outlines key branches of prior research and shows that a research gap exists in the area of
common examinations at the university level. The ability of standardized tests to powerfully motivate both
curriculum focus and instruction in elementary education has been established in previous work (Herman and Baker,
2005). Considerable work on teaching standardization followed passage of the No Child Left Behind Act, and led to
speculation of regarding college-level equivalents. Standardization concepts are particularly new in business
education, which differs from legal or medical studies because there is no general bar or licensure exam.

Pelimeni and Iorgulescu (2009) demonstrate that assessment has become “the new reality” on college and
university campuses. They show that although measuring student achievement is difficult, it is achievable if
instructors design assessment examinations using course objectives. They also suggest that customized but common
exams can be particularly helpful in college settings, where they serve as better assessment tools than standardized
tests.

Other research focuses on standardized tests and whether they improve pedagogy, skill recall, or in-depth
understanding of course material (Anaya, 1999 and Benjamin, 2008). Proponents suggest a collegiate level method
of evaluating student performance through standardized multiple-choice tests whose results would be used to grade
the overall performance of colleges. A suggested goal of their research is not just to improve testing, but rather to
improve pedagogy and curricula to create greater accountability in higher education. Examples include university
state initiatives in Oregon and Virginia (Cohen, 2009).

Much of this research is supported with further work by Klein (2005), who argues that it is possible to
assess cognitive outcomes and compare colleges and universities. Schmoker (2002) also suggests several simple
and replicable methods for improving teaching through substantial – but not exclusive – focus on assessed standards.
Creation of goals without such measureable baselines can lead to the implementation of unproven pedagogical and
teaching strategies. Fullan (2000) and Haycock (1998) also suggest that instructors and teachers meet regularly to
collectively focus on student work through common assessment in order to change instructional practice and achieve
better results. Bransford et al. (2002) supplement the above research by emphasizing the need to improve teaching
by employing methods to help students learn specific skills.

Literature in the field suggests a move towards standardization, but is still primarily focused on elementary
and secondary education. Those sources that address higher education, including Cohen (2009), include
descriptions of some college-level course assessment activities but leave a gap in the literature regarding professor
pedagogy, teaching incentive, and student outcomes. Specific research in business education, including work by
Abraham and Karns (2009) examines how business schools identify mastery of key competencies as an indicator of
successful graduates. It further compares those university-identified competencies to indicators that businesses
indentify with successful managers. While the research proposes stronger alignment between those identified
competencies, it does not include methods of improving the teaching or assessment of those skills.

To improve teaching and contribute to the field of study and literature, this research proposes a solution of
common examinations on a micro level. The goal is not only to create fair assessments, but to improve pedagogy in
the process by creating an incentive for better teaching.

METHODOLOGY

This study was conducted among undergraduate business majors of a mid-size (5000 - 10,000 students)
U.S. university. Results were drawn from a population of 245 college students who had different professors for the
same course. Examination and teaching results were compared among students taking the same quantitative
business course. Different classes ranged from 25-45 students, averaged 31 students per class, and were taught by
four different experienced professors. For the purposes of this study, where teaching performance was a major
variable, data from classes taught by the same professor were grouped together.

68
American Journal of Business Education – September 2010 Volume 3, Number 9
The test conditions of this study included a set of sections with equivalent student populations. Teaching
and sections had been standardized with common syllabi, common business skill prerequisites, and common study
materials, but outcome standardization depended on a common examination. Different professors taught common
content in all sections and administered an instructor-designed common final examination based exclusively on the
quantitative skills and competencies taught in the course. The exam design process began with a requirement and
agreement that one exam would be given to all students. The professors then cooperatively designed or chose
questions for inclusion on the common exam.

To assure valid comparisons, final examinations were assessed in a blind process, in which one grader
assessed all examinations with no knowledge of the name or class section of the students being graded. 85% of the
examination was objective and consisted of quantitative problems or multiple-choice questions. The remaining 15%
consisted of more subjective short answers or essays, but were assessed by a single grader using a common rubric to
assure uniformity.

RESULTS AND DISCUSSION

Comparing the data from a common assessment revealed important differences about what students learn,
and created opportunities and incentives to improve teaching practices. The results, as seen in Table 1, were
illuminating because of their dispersion. In evaluating the overall common final exam, the scores ranged from 13 to
95%. The median score was 57.6% and the mean was 58.5%. However, the performances among students under
different professors ranged markedly.

Table 1: Significant discrepancies spanning a common course
Class / Professor Mean Median Range
I 62.7 65.9 35-92
II 44.8 40.0 20-73
III 47.3 48.2 13-81
IV 71.3 75.3 47-95

After noting the obvious discrepancies (class II and III performances were significantly low,) professors
were able, for the first time, to clearly identify skill sets and competencies lacking among students in certain classes.
There was no notable correlation between individual class size and student performance. The public nature of the
differences uncovered an uncomfortable truth, and it motivated professors to examine reasons for the varying
performance levels of their students.

Public acknowledgement of differences or deficiencies can be a strong incentive for teaching improvement.
The common nature of a final exam created teaching accountability in a way that student evaluation and a tenure
system had previously failed to instill. As a result of the common exam, several changes were made to ensure that
all students had equal learning opportunities. While the goal was not to eliminate all grade discrepancies, the red
flag raised by significant differences was enough to prompt several changes and improved pedagogies. These
pedagogical changes included the adoption of additional problem-solving lessons, increasing lecture interactivity to
increase student engagement, the incorporation of case studies, and the performance of more in-class examples of
quantitative methods.
The common examination method suggested by these results differs from the standardized tests applied in
elementary education because the method used in this research suggests the need for instructor-designed exams.
Even common exams may be invalid if professors consider either the test content or test scoring to be inappropriate.
In this case, the exam scoring was uniform – both by grader and by rubric – and therefore the lack of variability left
little room for objection or charges of invalidity. However, the examination design requires the participation and
buy-in from the professors themselves, otherwise the common examinations may be subject to the argument that
they are asking the wrong questions.

The revelation of deficiencies suggested that earlier methods of teaching improvement were insufficient.
One of those earlier methods is the use of teaching evaluations, which many universities employ in an attempt to
69
American Journal of Business Education – September 2010 Volume 3, Number 9
provide signals to instructors and improve pedagogy. Generally, both student evaluations and peer evaluations of
instructors can provide useful suggestions, but evaluation confidentiality prevents public comparisons and hinders
accountability. A second earlier method for improving teaching was the incorporation of professors’ research
subjects into class material, which can potentially increase instructor interest, course applicability, and student
engagement. However Table 1 suggests that, if this is effective, it occurs unevenly. Thus the introduction of
instructor-designed common examinations complements, rather than supplants, existing methods of improving
teaching.

One important limitation of this research is that, although the results prompted recognition of teaching
deficiencies and spurred changes, the effects of those pedagogical modifications are not yet known. Future research
will gage the efficacy of the revised teaching methods. This will require examining both teaching and exam results
over several years to see which changes, if any, made a difference in student learning and performance. For that
future research, the current paper provides a baseline. The main benefit of the current research, however, is to
demonstrate how the introduction of common examinations can reveal the need for teaching modification and spur
both discussion and attempted improvements.

Several accrediting agencies suggest the creation, application, and public airing of a common final
examination based on specific competencies and skill sets. It is also a practice suggested by national benchmarking
legislation, although that legislation is limited to standardized exams during primary and secondary education. This
research suggests that university-level business education be improved with the inclusion of common, instructor-
designed testing. If this practice is adopted and if it reveals wide score ranges among classes, those diverse results
can prompt review, awareness, personal reflection, public discussion, and change.

CONCLUSION

The need for effective university teaching – particularly in business fields that demand both qualitative and
quantitative skills – requires innovative changes to improve pedagogy. This paper suggests that the introduction of
instructor-designed common exams can improve teaching because they reveal differences among professors that can
promote public accountability and therefore prompt improvement.

This research fills a gap left by prior research, which stressed accountability but focused on elementary
education. Those prior research papers that did examine university-level education focused on the value of tests for
assessing student outcome rather than for improving pedagogy.

This research began by noting the lack of incentives for improving university-level teaching and used
empirical data from business school student tests to propose common exams as a method of motivating improved
pedagogy. Results from almost 250 college students’ common assessments revealed important differences about
what students learn. This revelation, in turn, created opportunities and incentives to improve teaching practices
where other methods had failed earlier. The research in this paper suggests that common measurements can create a
metric for teaching accountability, which can in turn strengthen university teaching and programs.

AUTHOR INFORMATION

Dr. Darian Unger is an assistant professor at the Howard University School of Business who focuses on innovation
management, new product and technology development, and business education. He earned his Ph.D. from the
Massachusetts Institute of Technology and has published articles in the International Journal of Product
Development, the Journal of Management History, the Washington Business Research Journal, the Alliance Journal
of Business Research, and the Journal of Business and Economics Research.

REFERENCES

1. Abraham, S. and Karns, L., (2009), Do business schools value the competencies that businesses value?,
Journal of Education for Business, Vol. 84, Iss. 6, pp. 350-356.

70
American Journal of Business Education – September 2010 Volume 3, Number 9
2. Anaya, G., (1999), College impact on student learning: Comparing the use of self-reported gains,
standardized test scores, and college grades, Research in Higher Education, Vol. 40, Iss. 5, pp. 499-526.
3. Benjamin, R., (2008), The CLA contribution to improvement of teaching and learning in higher education,
Council for Aid to Education, 14 February, available at:
http://www.cae.org/content/pdf/The_CLA_Contribution_to_the_Improvement_of_Teaching_and_Learning
_in_Higher_Education.pdf (accessed May 20, 2010)
4. Bransford, J., Brown, A., and Cocking, R., (eds.), (2000), How People Learn: Brain, Mind, Experience,
and School: Expanded Edition, The National Academy Press, Washington D.C., pp. 131-189.
5. Cohen, A., (2009), No college student left behind?, The Futurist, Vol. 43, Iss. 3, pp. 13-14.
6. Cohen-Vogel, L., and McLendon, M., New Approaches to understanding federal involvement in education,
in Handbook of Education Policy Research, Edited by Sykes, G., Schneider, B., and Plank, D., American
Educational Research Association/Rutledge Publishers, New York, NY, pp. 735-748.
7. Figlio, D. and Kenny, L., (2007), Individual teacher incentives and student performance, Journal of Public
Economics, Vol. 91, Iss. 5-6, pp. 901-914.
8. Fullan, M., (2000), The three stories of education reform, Phi Delta Kappan, Vol. 81, Iss. 8, pp. 581-584.
9. Harris, D. and Sass, T., (2008), Teacher training, teacher quality and student achievement, Working Paper
No. 3, 12 March, National Center for Analysis of Longitudinal Data in Education Research (CALDER),
Washington DC, pp. 1-4.
10. Haycock, K., (1998), Good teaching matters: How well-qualified teachers can close the gap, Thinking 1-16,
Vol. 3, Iss. 2, p. 2.
11. Herman, J. and Baker, E., (2005), Making benchmark testing Work, Educational Leadership, Vol. 63, No.
3, pp. 48-54.
12. Hill, H., Rowan B., and Loewenberg Ball, D., (2005), Effects of teachers’ mathematical knowledge for
teaching on student achievement,” American Educational Research Journal, Vol. 42, Iss. 2, pp. 371-406.
13. Jepsen, C., (2005), Teacher characteristics and student achievement: Evidence from teacher surveys,
Journal of Urban Economics, Vol. 57, Iss. 2, pp. 302-319.
14. Kane, T. and Staiger, D., (2002), The promise and pitfalls of using imprecise school accountability
measures, Journal of Economic Perspectives, Vol. 16, Iss. 4, pp. 91-114.
15. Klein, S., Kuh, G., Chun, M., Hamilton, L., and Shavelson, R., (2005), An approach to measuring cognitive
outcomes across higher education institutions, Research in Higher Education, Vol. 46, No. 3, pp. 251-276.
16. Lavy, V., (2002), Evaluating the effect of teachers’ group performance incentives on student achievement,
Journal of Political Economy, Vol. 110, Iss. 6, pp. 1286-1317.
17. Marzano, R., Pickering, D., and Pollock, J. (2001), Classroom instruction that works: Research-based
strategies for increasing student achievement, Association for Supervision and Curriculum Development
(ASCD), Alexandria, VA, pp. 2-3.
18. Nye, B., Konstantopoulos, S. and Hedges, L., (2004), How large are teacher effects?, Educational
Evaluation and Policy Analysis, Vol. 26, Iss. 3, pp. 237-257.
19. Pelimeni, J. and Iorgulescu, R., (2009), Whose standards? Reaching the assessment puzzle, American
Journal of Business Education, Vol. 2, Iss. 7, pp. 43-53.
20. Reback, R. (2008), Teaching to the rating: School accountability and the distribution of student
achievement, Journal of Public Economics, Vol. 29, Iss. 5-6, pp. 1394-1415.
21. Schmoker, M., (2002), Up and Away, Journal of Staff Development, Vol. 23, No. 2, pp. 10-13.
22. Smith, H., (2008), Facing accountability in education: Democracy and equity at risk, The Journal of
Educational Research, Vol. 101, Iss. 6, pp. 379-380.

71
American Journal of Business Education – September 2010 Volume 3, Number 9
NOTES
72