ERIC EJ1140883: A Paradigm for Student Learning Outcome Assessment in Information Systems Education: Continuous Improvement or Chasing Rainbows?

0.38 MB·English

by  ERIC

Information Systems Education Journal (ISEDJ) 12 (1)
ISSN: 1545-679X January 2014
A Paradigm for Student Learning Outcome
Assessment in Information Systems Education:
Continuous Improvement or Chasing Rainbows?
Bruce Saulnier
[email protected]
Computer Information Systems
Quinnipiac University
Hamden, CT 06518, USA
Abstract
A paradigm is presented for student learning outcome assessment in information systems education.
Successful deployment of the paradigm is illustrated using the author’s home institution. The
paradigm is consistent with both the scholarship of teaching and learning and the scholarship of
assessment. It is concluded that the deployment of the paradigm allows us to address program
constituent concerns regarding student learning in higher education while simultaneously being
consistent with accreditation requirements at the program (ABET), school (AACSB) and institutional
(NEASC) levels.
Keywords: Assessment, Accreditation, Program Constituents, Program Educational Objectives,
Student Learning Outcomes.
1. INTRODUCTION career at graduation given both the catastrophic
student debt levels and the ever-changing
Over the course of the last decade there has economic landscape. All of these increasing
been an increased emphasis for student learning demands for accountability are arising at a time
outcome assessment at the national level. of both a decreasing traditional college-age
Although “No Child Left Behind” (United States population and the emergence of Massive Open
Congress, 2002) and “Race to the Top” (United Online Courses (MOOC’s).
States Department of Education, 2009) have
garnered the most press, this national In response to these increased demands for
movement for educational accountability is now accountability institutions of higher education
directly impacting accreditation requirements of have placed an increased emphasis on
the regional higher education accrediting assurance of learning by measuring student
agencies (NEASC, 2011). Additionally, both learning outcomes. In many cases desired
school accreditation requirements (AACSB, student learning outcomes have been defined at
2003) and program accreditation requirements the university, school, and program level. But
(ABET, 2012) have issued calls for the are we in higher education simply chasing
assessment of student learning. rainbows? Can student learning be measured, in
any real sense of the word? The fallout from “No
Further impacting the need for higher education Child Left Behind” is legendary, and the “Race to
institutions to address assurance of learning is the Top” is increasingly being met with
the public’s demand for proof that graduates will skepticism. But if we in higher education fail to
have a reasonable opportunity for a successful respond to the increasing demands for
©2014 EDSIG (Education Special Interest Group of the AITP) Page 4
www.aitp-edsig.org /www.isedj.org
Information Systems Education Journal (ISEDJ) 12 (1)
ISSN: 1545-679X January 2014

accountability with measures and processes that agencies, devotes standards 4.48 through 4.54
are meaningful to us, and if we fail to convince to the assessment of student learning,
the public that our results are meaningful, then specifically requiring that each academic
it is likely that the measures and processes will institution implements and provides support for
be defined for us. systematic and broad-based assessment of how
students are learning. NEASC further requires
2. INFORMATION SYSTEMS that each institution use a variety of quantitative
ACCREDITATION REQUIREMENTS and qualitative methods and both direct and
indirect measures to understand the experiences
For undergraduate programs in Information and learning outcomes of its students, and that
Systems, accreditation requirements exist at the institution use the results of these
least at two levels: (1) regional accreditation assessments for improvement.
agencies, for which requirements must be met;
and (2) at the program level, through ABET, At the school level, our information systems
which is a program-level option. Additionally, for program is located in an AACSB-accredited
programs existing within Schools of Business, school of business, and as such our school-wide
accreditation requirements exist through the accreditation must conform to AACSB Assurance
Association to Advance College Schools of of Learning standards. These standards are
Business (AACSB), although this is also a based on the premise that learning is the central
voluntary option. All three levels of accreditation activity of higher education, and that the
require attention to the assessment of student definition of learning expectations and assurance
learning, though the individual requirements that graduates achieve learning expectations are
vary in terms of the language they employ. key features of any academic program. AACSB
Standard 16 specifically requires that for each
While much prior work has been done in terms undergraduate degree program the school must
of information systems assessment, and this define learning goals, and that for each
prior work has appeared in major Information academic program the school demonstrates that
Systems (IS) journals, only a few articles students meet the learning goals. Moreover, if
(Beard, Schweiger & Surendran (2008); Mills, assessment demonstrates that learning goals
Hauser, & Pratt (2008)) appear to link IS are not being met, that processes are in place
assessment to larger issues of school-wide and are being employed to eliminate the
assessment. An exhaustive literature such has discrepancy.
failed to turn up a single article that links IS
assessment to larger institutional assessment At the program level, we choose to use ABET
concerns associated with regional accreditation. guidelines to maintain program-level
Further, most of the current literature is micro in accreditation, which requires that student
its scope advocating for either (1) a particular performance be monitored to foster success in
method associated with a particular course attaining student outcomes, thereby enabling
and/or learning outcome or (Carpenter, Snyder, graduates to attain program educational
Slauson, & Bridge (2011); Murray, Perez, & objectives. As such, we must define our program
Guimaraes (2008); Wagner, Longenecker, level constituencies consistent with the ABET
Landry, Lusk, & Saulnier (2008)) or (2) the definition of constituency, define our Program
effectiveness of a particular method employed Educational Objective, define our student
across the IS curriculum (Al-Mubaid, Learning Outcomes necessary for our students
Abeysehera, Kim, Perkins-Hall, & Yue (2011); to obtain program educational objectives, and
AAsheim, Gowan, & Reichgelt (2007); Saulnier, develop and execute a successful assessment
Landry, Longenecker, & Wagner (2008)). But program to insure that our program is meeting
none of the work to date has focused on the its educational objectives and modify it as
larger issue of providing a paradigm that necessary based on assessment results.
addresses the link of assessment to IS ABET
program accreditation while simultaneously 3. PRECURSOR/PRINCIPLES FOR
addressing assessment requirements at both the EFFECTIVE STUDENT LEARNING
school and institutional-levels.
In response to the need to address how effective
At the institutional level, the New England our students are learning, it is desirable for the
Association of Schools and Colleges (NEASC), faculty to engage in scholarly teaching; that is,
one of the six major regional accrediting whatever teaching and assessments they

©2014 EDSIG (Education Special Interest Group of the AITP) Page 5
www.aitp-edsig.org /www.isedj.org
Information Systems Education Journal (ISEDJ) 12 (1)
ISSN: 1545-679X January 2014

employ should be consistent with what we know best practices to information systems programs
about how students learn. While the Scholarship attempting to meet these requirements. Rather
of Teaching and Learning (SoTL) has been an than conduct separate assessments to meet
object of higher education research for decades, separate accreditation requirements at the
the last decade has added significantly to our university, school, and program levels, it is
knowledge base. highly desirable to employ an integrated
procedure that meets all three sets of
Building on the earlier work of Chickering & requirements in a single process. It is further
Gamson (1987), Bransford (2000) has provided desirable that the integrated process should
significant insight into the science of learning. promote the use of the AAC&U high impact
Consistent with Bransford’s findings, Fink (2003) practices consistent with the backwards course
asks us to move beyond the earlier taxonomy of design espoused by Fink.
Bloom (1956) to produce significant learning for
our students by engaging in backward course 5. THE PARADIGM
design. Kuh (2008), writing on behalf of a
nationally commissioned study group of the The following 7-step process has been used to
American Association of Colleges and design and develop a program assessment and
Universities (AAC&U), extends the paradigm continuous improvement system consistent with
further by categorizing certain pedagogies as the accreditation requirements of NEASC,
High Impact Practices; that is, a research-based AACSB, and ABET:
group of instructional practices that have been
shown to positively impact student learning. Step 1. Establish Program Constituencies
Although ABET does not specifically define what
We are fortunate at Quinnipiac University to be it means by a constituency of the program, ABET
in an academic environment that has made an requires that the program must have published
intentional commitment to become an exemplar educational objectives that are consistent with
of a Learning Paradigm College (Tagg, 2005). As the mission of the institution, the needs of the
such, all institutional resource allocation program’s various constituencies, and the ABET
decisions are made based on the degree to criteria for accrediting computing programs. In
which they have the potential to positively other words, a program’s educational objectives
impact student learning (Barr & Tagg, 1995). are based on the needs of the constituencies. It
Indeed, our program focuses on active “learning is therefore necessary for a program to have
by doing” instructional practices, and among the defined constituencies who are consulted to
ways we actively address student learning are establish the program’s educational objectives.
our Information Technology for Good (IT4G) While the ultimate responsibility for curriculum
initiative, a commitment to service learning, must necessarily lie with the program faculty
project-based courses with real projects (usually (NEASC, 2011), the definition of program
service learning projects done for not-for- educational objectives is made with the input
profits), and required internships. and concurrence of the program’s
constituencies.
4. PROBLEM STATEMENT
Our CIS program has defined our program
All of us in higher education in general, and in constituencies to be (1) the full-time program
Information Systems in particular, seek to faculty, (2) the CIS Advisory Board, (3) Alumni
promote continuous improvement in our of the program, and (4) employers of our
curriculum that results in improved student graduates. The purpose of a constituency, then,
learning. But in order to objectively do so, we is really to assist in the definition of program’s
need a mechanism to “measure” our students’ educational objectives.
learning. Such has been the driver for the
emergence of assessment requirements at the Step 2. Define Program Educational
regional, school, and program levels. Objectives
Program Educational Objectives (PEO’s) are by
While much has been done to promote and definition broad statements that describe what
advance both continuous improvement and graduates are expected to attain within a few
assessment of learning accreditation years after graduation (ABET, 2012). We have
requirements, little appears to have been done interpreted a “few years” to mean 3-5 year goals
to provide guidance and/or promote the use of of the program. Specific PEO’s were adopted by

©2014 EDSIG (Education Special Interest Group of the AITP) Page 6
www.aitp-edsig.org /www.isedj.org
Information Systems Education Journal (ISEDJ) 12 (1)
ISSN: 1545-679X January 2014

a vote of the full-time program faculty after (1) (f) An ability to communicate with a range
presentation to and feedback from the of audiences;
program’s advisory board, (2) interviews with (g) An ability to analyze the local and global
internship supervisors and employers of our impact of computing on individuals,
graduates, and (3) a formally conducted organizations, and society;
program alumni survey. (h) Recognition of the need for and an
ability to engage in continuing
Specific PEO’s adopted by our CIS program are professional development; and
within three-five years of graduation program (i) An ability to use current techniques,
graduates will have: skills, and tools necessary for computing
1. Helped an organization achieve its goals practice.
by applying knowledge and skills in the (j) An understanding of the processes that
application of information systems; support the delivery and management of
2. Used information systems for decision information systems within a specific
making to help organizations achieve a application environment.
strategic competitive advantage;
3. Served as liaisons between end-users These specific outcomes (a)-(i) are
and computing specialists by recommended/viewed by ABET as the minimal
communicating effectively in both oral set of required LO’s. While individual programs
and written form; are free to adopt additional LO’s to support their
4. Worked effectively in teams to manage PEO’s and the specific needs of their
both themselves and their colleagues; constituencies, they are not required to do so.
and
5. Demonstrated lifelong learning skills by Though there is no mandate that the ABET LO’s
attendance at continuing professional by discussed with the program constituencies,
education courses/workshops, pursuit we did so to obtain their input as to the
and/or attainment of professional appropriateness and completeness of the
certification, and/or higher-level recommended ABET list. After this discussion,
academic education. the faculty of the department unanimously
adopted the ABET list as our approved student
Step 3. Define Student Learning Outcomes learning outcomes.
ABET accreditation requirements (ABET, 2012)
specify that the program must have documented Step 4. Map LO’s onto PEO’s
student learning outcomes (LO’s) that prepare To determine whether the list of LO’s contribute
graduates to attain the program educational to student attainment of all of the PEO’s a
objectives, and that there must be a mapping of the LO’s onto the PEO’s is
documented and effective process for the constructed. This insures that the LO’s are
periodic review and revision of these student sufficient to attain the desired PEO’s. The
outcomes. The requirements specifically (p. 3) specific mapping of our LO’s onto our PEO’s is
specify that the program must enable students shown in Exhibit 1:
to attain, by the time of graduation:
(a) An ability to apply knowledge of Mapping of LO’s Onto PEO’S
computing and mathematics appropriate (Learning Outcome Contribution to PEO’s)
to the discipline;
(b) An ability to analyze a problem, and LOs/PEOs PEO 1 PEO 2 PEO 3 PEO 4 PEO 5
identify and define the computing LO-a X
requirements appropriate to its solution; LO-b X X
(c) An ability to design, implement, and LO-c X X
evaluate a computer-based system, LO-d X X
process, component, or program to meet LO-e X X
desired needs; LO-f X X X
(d) An ability to function on teams to LO-g X
accomplish a common goal; LO-h X
(e) An understanding of professional, LO-i X X
ethical, legal, and security and social
LO-j X X
issues and responsibilities;

©2014 EDSIG (Education Special Interest Group of the AITP) Page 7
www.aitp-edsig.org /www.isedj.org
Information Systems Education Journal (ISEDJ) 12 (1)
ISSN: 1545-679X January 2014

Exhibit 1 … Mapping LO’s Onto PEO’s student. Hence, we have mapped but two LO’s
onto the required internship.
Step 5. Assign LO’s to Specific Required
Courses Our CIS Alumni Survey, our CIS Advisory Board
Once we are convinced that our LO’s are Input, and the Career Leader Assessment
sufficient to attain our desired PEO’s, the next survey/test administered in SB 112 Career
step is to assign responsibility for delivering Development are the means via which we
learning outcomes to particular courses. Given receive input and document our effectiveness in
that our students will not necessarily take achieving outcome h – recognition of the need
elective major courses, it is necessary that all for continuing professional development. This
learning outcomes be addresses in required outcome is further reinforced by the internship
major courses. While some learning outcomes supervisor input regarding students’ attitudes
may also be covered in courses outside the and experiences in CIS 484.
major, we cannot necessarily control what is
being taught in those courses. We administer the Information Systems Analyst
(ISA) exam in CIS 440, though the information
Required courses specific to our program in covered on the ISA exam is not necessarily
Computer Information Systems (CIS) are as covered in CIS 440; (please see particulars in
follows: CIS Assessment Plan – Appendix A).
CIS 125 Systems Analysis & Design
CIS 225 Object-Oriented (OO) SAD The LO’s that have been assigned to specific
CIS 244 OO Programming courses become the basis for both course design
CIS 301 Enterprise Systems consistent with the principles of backward course
CIS 330 Networking & Data Com. design (Fink, 2003), and the adoption of high
CIS 351 Database Design & Prog. impact practices (Kuh, 2007) to address the
CIS 440 IT Project Management assigned LO’s specific to each required course.
CIS 484 IT Internship While courses may have individual learning
objectives beyond those that appear in the
Each of these required courses is specifically Course Responsibility Matrix, the matrix
assigned responsibility for one or more of the becomes the driver for a minimal set of learning
LO’s within the framework of Bloom’s taxonomy outcomes for each required course.
of educational objectives (Bloom, 1956). The
specific mapping of our LO’s onto our required Step 6. Adopt an Assessment Plan to
courses is shown in Exhibit 2. Monitor Attainment of LO’s
Prior to constructing individual course syllabi and
L0’S/ CIS CIS CIS CIS CIS CIS CIS CIS developing course assignments to specifically
Course 125 225 245 301 330 351 440 484
address assigned LO’s, an assessment plan
LO-a X X
should be developed such that both the course
LO-b X X X
syllabus and course assignments are constructed
LO-c X X
consistent with departmentally approved
LO-d X X
assessment criteria. Such a plan should
LO-e X X
minimally indicate specifically, for each LO, the
LO-f X X X
following information:
LO-g X
 Where and with what frequency the
LO-h X outcome would be assessed;
LO-i X X  The specific assessment methods for
LO-j X X each instantiation;
Exhibit 2 Course Responsibility Matrix  The a priori target criteria for student
(Mapping LO’s onto Required CIS Courses) performance to be deemed satisfactory;
 Specific assessment results that will be/
CIS 484 is a required internship which is have been obtained from the execution
supervised by company personnel. While we are of the plan;
comfortable that success in this internship  Document which specific actions have
experience is a highly professional capstone been or will be taken as a result of the
experience, we cannot necessarily document the evaluation of assessment results.
experience across a range of LO’s for every

©2014 EDSIG (Education Special Interest Group of the AITP) Page 8
www.aitp-edsig.org /www.isedj.org
Information Systems Education Journal (ISEDJ) 12 (1)
ISSN: 1545-679X January 2014

The adopted plan can/should be a combination Supervisors consistent with the
of both direct and indirect assessment requirements of our Assessment Plan;
measures/methods. The field of higher education  Available data was collected from prior
assessment has been an area of scholarly semesters CIS required courses to
inquiry for decades, and we should avail provide a baseline and effectively
ourselves of the scholarly body of knowledge summarize results relative to our
about assessment to effectively construct such a adopted plan; and
plan. From the seminal works of Astin (1991)  Data was collected from all of the School
and Angelo & Cross (1993) to the more recent of Business courses that contribute to
contributions of Suskie & Banta (2009) and our assessment plan.
Sambell, McDowell & Montgomery (2013), much
is known about effective assessment techniques The faculty of the department met during
and strategies that are consistent with and January 2013 to discuss the results of the data
complimentary to what we know about the most gathered to date and plan for changes that were
effective teaching methods and how people implemented during the spring 2013 semester.
learn. While standardized testing has a place in Further assessment results were obtained from
higher education, the use of such tests can and spring 2013 courses and this data has been
should be primarily formative/diagnostic, not the discussed by the faculty during their end of the
common summative evaluative culture of the year meeting to plan for appropriate changes for
academy. the fall 2013 semester.

The CIS Department adopted assessment plan is 6. RESULTS
included as information in Appendix A. The plan
was constructed to be consistent with both The adopted paradigm has been employed
NEASC reporting requirements via consultation during the 2012 ABET reaccreditation process.
with our campus-wide assessment coordinator, Although the results of the reaccreditation
and with AACSB reporting requirements via process are not known at the time of this paper
consultation of with the associate dean of our submission, the process has been well received
school of business. While readers may employ by the members of the ABET site visitation
the format of the plan for guidance, it is highly team.
advisable that each department‘s faculty
construct their own plan in consultation with the The paradigm itself is consistent with the
appropriate individuals on their respective measures required for both NEASC institutional
campuses. accreditation and AACSB school of business
reaccreditation, and results to date are
Step 7. Implement the Assessment Plan consistent with both NEASC and AACSB
The assessment plan was implemented during requirements.
the 2012 calendar year with data collected from
both the spring 2012 and fall 2012 semesters. A very real benefit of the deployment of this
Syllabi were constructed and course-embedded paradigm has been the almost universal
assessment measures were adopted consistent adoption of backwards course design principles
with the learning outcome responsibilities by the department faculty. That is, course syllabi
associated with each required course. are constructed to support student learning
outcomes associated with the course, in-class
In addition to course-embedded assessments, activities are designed to support the syllabi,
the following activities were undertaken assignments are constructed consistent with
consistent with the requirements of the desired learning outcomes, and assessment
assessment plan: measures are adopted which specifically address
 An alumni survey was conducted to course learning outcomes. Consequently,
ascertain alumni opinions of both the student performance is rising as we become
PEO’s and LO’s, the effectiveness of our much more intentional in our teaching and
program, and other factors needed to learning focus.
measure our LO’s consistent with our
adopted Assessment Plan; 7. CONCLUSIONS
 Individual interviews were conducted
with both the members of the CIS The presented paradigm has been developed
Advisory Board and CIS Internship and deployed and has yielded results that are

©2014 EDSIG (Education Special Interest Group of the AITP) Page 9
www.aitp-edsig.org /www.isedj.org
Information Systems Education Journal (ISEDJ) 12 (1)
ISSN: 1545-679X January 2014

consistent with ABET program reaccreditation http://www.aacsb.edu/accreditation/busines
requirements, AACSB school-wide assessment s/standards/aol/ .
requirements, and university regional
accreditation requirements. Aasheim, J., Gowan, J.A., and Reichgelt, H.
(2007). Establishing an Assessment Process
A major benefit of intentional involvement in the for a Computing Program. Information
accreditation and assessment processes is that it Systems Education Journal, 5 (1).
drives department faculty to consider questions http://isedj.org/5/1/. ISSN: 1545-679X
and issues that they ought to be considering on
a regular basis, but frequently get overlooked ABET (2012). Criteria for Accrediting Computing
during the rapid rhythms of the normal semester Programs 2012-2013. Retrieved May 10,
activities. In particular, construction of the 2013 from
assessment matrix forces an intentional http://www.abet.org/DisplayTemplates/Docs
consideration of student learning as the primary Handbook.aspx?id=3142 .
driver of course design.
Al-Mubaid, H., Abeysekera, K., Kim, D., Perkins-
The major limitation of the paradigm is not in Hall, S., Yue, K. (2011). A Model for Long
the design of the paradigm, but rather in its Term Assessment of Computing and
implementation. The outcome measures Information Systems Programs. Information
developed to date are predominantly indirect Systems Education Journal, 9(3) pp 59-67.
measures of student learning because successful http://isedj.org/2011-9/ ISSN: 1545-679X
exam performance does not necessarily mean
that students can effectively perform the tasks Angelo, T. and Cross, K.P. (1993). Classroom
in a professional work environment. While the Assessment Techniques: A Handbook for
use of service-based projects provides for better College Teachers. San Francisco: Jossey-
measures of student performance, those Bass.
measures are very difficult to quantify.
Astin, A. (1991). Assessment for Excellence: The
So we return to the title of this article. Are we Process and Practice of Assessment and
just chasing rainbows? We think not! The Evaluation in Higher Education. New York:
deployment of this paradigm has provided the American Council on Education.
CIS department with reliable high quality data to
provide to stakeholders concerning the learning Barr, R. and Tagg, J. (1995). From Teaching to
of our CIS students while simultaneously Learning: A New Paradigm for
addressing the needs of our program Undergraduate Education. Change: 27:6.
constituencies, and it has done so in a manner
that supports both school-wide and university- Beard, D., Schweiger, D., Surendran, K. (2008).
wide assessment and accreditation Integrating Soft Skills Assessment through
requirements. Further, maintenance of University, College, and Programmatic
accreditation at the program level provides the Efforts at an AACSB Accredited Institution.
department with data to support ongoing Journal of Information Systems Education,
preferential treatment in terms of budget 9(2) pp. 229-240.
allocation to support faculty professional
development activities and student learning Bloom, B. S. (1956). Taxonomy of Educational
outcomes. Objectives, Handbook I: The Cognitive
Domain. New York: David McKay Co Inc.
One final point – the adoption of this paradigm
has directed the faculty toward a much more Bransford, J., Brown, A. & Cocking, R. (2000).
intentional focus on the learning of our students. How People Learn: Brain, Mind, Experience,
In the final analysis, isn’t that what our courses and School. Washington, DC: National
should really be about? Academy Press.

8. REFERENCES Carpenter, D., Snyder, J. E., Slauson, G. ..,
Bridge, M. (2011). Additional Support for the
AACSB International (2003). 2003 Business Information Systems Analyst Exam as a
Accreditation Standards: Assurance of Valid Program Assessment Tool. Information
Learning. Retrieved May 10, 2013 from

©2014 EDSIG (Education Special Interest Group of the AITP) Page 10
www.aitp-edsig.org /www.isedj.org
Information Systems Education Journal (ISEDJ) 12 (1)
ISSN: 1545-679X January 2014

Systems Education Journal, 9(4) pp. 73-79. Accreditation. Retrieved May 10, 2013 from
http://isedj.org/2011-9/ ISSN: 1545-679X http://cihe.neasc.org/standards_policies/sta
ndards/standards_html_version/.
Chickering, A. W., & Gamson, Z. F. (1987).
Seven Principles for Good Practice in Sambell, K., McDougall, L., and Montgomery, C.
Undergraduate Education. AAHE Bulletin, 3- (2013). Assessment for Learning in Higher
7. Education. Oxon, England: Routledge.

Fink, L. D. (2003). Creating Significant Learning Saulnier, B., Landry, J., Longenecker, B.,
Experiences: An Integrated Approach to Wagner, T. (2008). From Teaching to
Designing College Courses. San Francisco: Learning: Learner-Centered Teaching and
Jossey-Bass. Assessment in Information Systems
Education. Journal of Information Systems
Kuh, G. (2008). High-Impact Educational Education, 9(2) pp. 169-174.
Practices: What They Are, Who Has Access
to Them, and Why They Matter. Washington, Suskie, L. and Banta, T. (2009). Assessing
DC: AAC&U. Student Learning: A Common Sense
Approach. San Francisco: Jossey-Bass.
Mills, R., Hauser, K., Pratt, J. (2008). A
Comprehensive Two-Level Framework for Tagg, J (2003). The Learning Paradigm College.
Information Systems Curriculum Design, San Francisco: Jossey-Bass.
Assessment, and Improvement. Journal of
Computer Information Systems, 48(4) pp. 1- United States Congress (2002). PL 107-110. The
14. No Child Left Behind Act of 2001. Retrieved
May 8, 2013 from
Murphy, M. C., Sharma, A., Rosso, M. (2012). http://www2.ed.gov/policy/elsec/leg/esea02
Measuring Assurance of Learning Goals: /index.html.
Effectiveness of Computer Training and
Assessment Tools . Information Systems United States Department of Education (2009).
Education Journal, 10(5) pp. 87-94. Race to the Top. Retrieved May 8, 2013 from
http://isedj.org/2012-10/ ISSN: 1545-679X. http://www2.ed.gov/programs/racetothetop
/index.html.
Murray, M, Perez, J., Guimaraes, M. (2008). A
Model for Using a Capstone Experience as Wagner, T., Longenecker, B., Landry, J., Lusk,
One Method of Assessment of an S., Saulnier, B. (2008). A Methodology to
Information Systems Degree Program. Assist Faculty in Developing Successful
Journal of Information Systems Education, Approaches for Achieving Learner Centered
9(2) pp. 197-208. Information Systems Curriculum Outcomes:
Team Based Methods. Journal of Information
NEASC: Commission on Institutions of Higher Systems Education, 9(2) pp. 181-186.
Education (2011). Standards for

Editor’s Note:
This paper was selected for inclusion in the journal as an ISECON 2013 Distinguished Paper. The
acceptance rate is typically 7% for this category of paper based on blind reviews from six or more
peers including three or more former best papers authors who did not submit a paper in 2013.

©2014 EDSIG (Education Special Interest Group of the AITP) Page 11
www.aitp-edsig.org /www.isedj.org
Information Systems Education Journal (ISEDJ) 12 (1)
ISSN: 1545-679X January 2014
Appendix A - CIS ASSESSMENT PLAN

Recommendations
Student Where Assessment A-priori Target Assessment for Improvement /
Learning Assessed/ Methods Criteria Results Documentation
Outcomes How
Often
Assessed
a. An ability to 80% of students . Meets criteria.
apply knowledge CIS 245 / Programming will score 75% + 80% of Continue current
of computing yearly Assignments on 80% of students assignments and
and programming scored >= assessments.
mathematics assignments. 75% or above
appropriate to CIS 330 / Networking 90% of students on all Meets criteria.
the discipline;* yearly Assignments will score 75% programming Continue current
+ on 75% of assignments assignments and
networking 95% of assessments.
assignments students
EC 271 / EC 271 scored above Meets criteria.
Each common CIS Students 75% on all Continue with
semester statistics final will have an course current focus.
exam average score of networking
75% + in assignments
common exam Below average.
SB 450 / ETS exam – in EC 271. Average score Area of concern.
Each Quant Bus Anal on EC 271
Semester / yearly CIS students will final exam
have an average was 83%
score of 50% +
on the ETS
Statistics CIS ETS
questions Quant Avg. =
44%
b. An ability to Meets criteria.
analyze a CIS 125 / “Requirements” 125 - 90% of 125 - 96% Continue current
problem, and yearly Assignment students will scored assignments and
identify and score satisfactory or assessments.
define the Course satisfactory or higher
computing CIS 225 / assignments higher 225 – above target,
requirements yearly 225 - 80% + on 225 – 88% but need
appropriate to its ISA Exam 80% of SAD scored above additional work on
solution; CIS 440 / (Systems assignments in 80% activity modeling
yearly Analysis) / 225 QU average ISA - Meets
Yearly CIS majors will was 63% in criteria
have average 2010 ISAE
score 60% + in section
Systems
Analysis section.
c. An ability to Meets criteria.
design, CIS 245 / Programming 80% of students 80% of Continue current
implement, and yearly assignments will score 75% + students assignments and
evaluate a on 80% of scored >= assessments.
computer-based programming 75% on all

©2014 EDSIG (Education Special Interest Group of the AITP) Page 12
www.aitp-edsig.org /www.isedj.org
Information Systems Education Journal (ISEDJ) 12 (1)
ISSN: 1545-679X January 2014
system, process, assignments. programming
component, or CIS 351 / Database 90% of students assignments Meets criteria,
program to meet yearly assignments will score 75% continue with
desired needs;* + on database 90% of current
projects students assignments and
scored at assessments.
85.4%
d. An ability to CIS 351 / In-course team 90% of students 90% of 351 - Meets
function yearly project score 75% or students criteria, continue
effectively on above on team scored at with current
teams to rubric 85.4% assignments
accomplish a CIS 440 / In course team 440 – Meets
common goal;* yearly project 440 – All teams 440 – 86% Criteria
score 75% or scored at
above 86%; 14%
scored below
50%

e. An CIS 301 / In-course 301 – 85% > 301 – 88% 301 - Meets criteria
understanding yearly assignment: 90% scored 90%+ 440 – below
of professional, CIS 440 / Write Ethics 440 – 85% > 440 – 86% > criteria, 3 students
ethical, legal, yearly Report 90% 90% basically no show
security, and skew results
social issues and SB 450 / ETS Exam – ETS Exam Avg. CIS ETS ETS – Meets
responsibilities;* yearly Legal & Social > 50% Avg. Score = Criteria
/ yearly 59%
f. An ability to CIS 125 / In course 90% of students 92% scored 125 - Meets criteria
communicate yearly presentation will score superior;
effectively with a and analysis satisfactory or 100% scored 301 – Meets
range of CIS 301 / In course higher satisfactory or criteria
audiences;* yearly assignments 90% adequate higher
and 301 – 95% 484 – paper meets
CIS 484 / presentations 90% of CIS 484 Adequate or criteria
yearly Company Internship form better
Analysis Paper respondents 484 – 100%
must grade QU of papers Survey – meets
Internship CIS students criteria
Supervisor Internship “adequate/above Internships -
Survey feedback / average” in 100% grade
yearly communication. outstanding
or above
average
g. An ability to CIS 125 / In-Course 80% score 96% scored 125 – Meets
analyze the local yearly assignment satisfactory or satisfactory or Criteria
and global assessed with higher on higher; 76%
impact of common rubric assignment at superior
computing on SB 450 ETS Exam – related to ETS – Meets
individuals, International / local/global Criteria
organizations, yearly impact CIS Avg. =
and society;* CIS Avg. > 50% 62%
h. Recognition SB 112 Career Leader Avg. CIS Score AVG CIS 112 – Meets
of the need for Alumni Assessment on Creative, Score > 75% Criteria for all
and ability to Survey / Alumni Survey; Critical, and for all 3 sub- three subsections
engage in every 3 Strategic sections

©2014 EDSIG (Education Special Interest Group of the AITP) Page 13
www.aitp-edsig.org /www.isedj.org