ERIC ED339438: Impact of Student Educational Outcomes Assessment on Virginia Institutions of Higher Education.

9 Pages·1991·0.55 MB·English

by  ERIC

DOCUMENT RESUME
JC 910 580
ED 339 438
Darrel A.
Levin, Bernard H.; Clowes,
AUTHOR
Outcomes Assessment on
Impact of Student Educational
TITLE
Education.
Virginia Institutions of Higher
91
PUB DATE
9p.
NOTE
(143)
Reports - Research/Technical
PUB TYPE
MF01/PC01 Plus Postage.
EDRS PRICE
*College Outcomes
Administrator Attitudes;
DESCRIPTORS
*Government School
Assessment; Community Colleges;
*Outcomes of
Relationship; Higher Education;
School Surveys;
Education; Program Effectiveness;
Attitudes; *Teacher
*State Standards; Teacher
Behavior; Two Year Colleges
Assessment VA;
*Student Educational Outcomes
IDENTIFIERS
*Virginia
ABSTRACT
institutions in
For the past 4 years, educational
educational
mandate to conduct student
Virginia have been under state
reports to the
(SEOA) and to provide periodic
outcomes assessment
effort to
Education in Virginia. In an
State Council of Higher
institutions of
developed within Virginia's
examine how SE0A has
mailed to the faculty senate
higher education, a survey was
chief assessment
committee chairpersons, and
chairpe7sons, curriculum
and
35 public senior institutions,
officer of 23 community colleges,
rates were 62%.
in the state. Response
28 private senior institutions
different types of institutions
26%0 and 27% for the three
39%. Specifically,
overall response rate of
respectively, yielding an
full-time faculty
on the percent of
the survey sought information
institution's
the impact of SEOA on the
directly involved in SEOA,
into academic
the integration of SEOA
povernance structure, and
of
the following: (1) estimates
practices. Study findings included
instituitons, 45% in
38% in public senior
faculty involvement were
caleges; (2)
and 51% in the community
private senior institutions,
type of
indicated that SECA had led to any
only 15 of 74 responses
respondents at public senior
structural change; (3) 57% of
senior institutions, and 51% at
institutions, 43% at private
normal
had been integrated into
community colleges believed SEOA
least likely
senate chairpersons were
academic processes; (4) faculty
that SEOA
(5) open-ended comments indicated
to respond to the survey;
findings
administrative process. Study
had become predominately an
anything
has yet to prove itself as
suggest that SEOA in Virginia
References are included. (PAA)
other than a reporting process.
*****01*******O******************************M*****z*****************0
the best that can be made
Reproductions supplied by EDRS are
from the original document.
********************** ***** ******************
******************K** *****
impact of Student Educational Outcomes Assessment on Virginia
Institutions of Higher Education
Bernard H. Levin and Darrel A. Clowes
U
Si DEPARTMENT Of EDUCATION
"PERMISSION TO REPRODUCE THIS
/ft. 4. of f dui *Omer Posner, h arid imixormeni
MATERIAL HAS BEEN GRANTED BY
fiKSTrONAL. RESOURCES ME ofmTioni
CE Pit TER a RIC)
B. Levin
tfus document has helm reproi,00 as
oronaston
'erstsisli from 1114 mown
tOrglnalsrig
hare We."
Mmv+
rr.cfrr n improve
woodut hon issihfr
.
.
.
Peonts at roe* or Opr V(Ntle Ste:e.C,,n TrVO (NW V
TO THE EDUCA1IONAL RESOURCES
rnprd do iii n0,Santy ropresnt otta
INFORMATION CENTER tERIC)
(14 Pr posrliorr or pohr y
9
AVAILABLE
BEST COPY
Impact of Student Educational
Outcomes Assessment on Virginia
Institutions of Higher Education
Bernard H. Levin1 and Darrel A. Clowes"
Institutions of higher education
seem vulnerable to forces
exerted by the larger society.
For example, during the late 1960s
and early 1970s we emphasized behavioral
objectives both in
administration and instruction. Then
we experienced frenzies of
"evaluation
of
teaching."
recent
In
years,
"institutional
effectiveness" has become
a byword in higher education [e.g.,
Southern Association of Colleges and Schools,
1988]. No operational
definition of institutional
effectiveness has been universally
adopted. In many institutions it
has translated [often via state
mandate] to Student Educational
Outcomes Assessment [SEOA].
Two broad questions
are worthy of examination:
Does
institutional
1.
effectiveness
affect
higher
education institutions differentially?
2. What is the effect of SEOA
on administration, faculty,
academic programs,
and governance structures within
institutions of higher education
and what is the effect
upon external agencies and institutions?
Institutional effectiveness is
an ill defined concept. Welker
[1990] has traced the literature
on institutional effectiveness and
higher education from 1970 to
1990. He found it to peak in the late
1970's for all of higher education
and thereafter to gradually
decline. All sectors of higher education
are represented similarly
in the literature until
the late 1980's when community college
literature appears disproportionally
involved with considerations
of institutional effectiveness.
Welker found that the literature
on effectiveness has almost
no basis in auipirical research; it is predominantly
a descriptive
and prescriptive literature and is,
in our view, more properly
labelled a literature of management efficiency.
Welker's work also
implies that institutional effectiveness
is more a concern in the
community colleges than in other
sectors of higher education.
Student educational outcomes assessment is
a logical extension
of the institutional effectiveness
movement.
In fact, the concepts
are mingled in the Southern Association
of Colleges and Schools
literature.
Welker [1990] also found a strong inter-relationship.
We would expect SECA to follow
the pattern of the institutional
effectiveness movement and impact
most heavily upon the community
college.
The community college has the least
secure place within
higher education 1Clowes & Levin,
1989] and is the most vulnerable
to external pressures asserted
through state coordinating and
controlling agencies and state
legislatures.
Matlick [1990] surveyed selected
facu]ty and administrators in
Maryland's public two and four
year colleges to determine their
attitudes toward and perceptions
of SEOA's impact.
She reported
3
Levin & Clowes, p.
2
by
impact
perceived
and
attitude
in
differences
systematic
personnel were generally
institutional type. Both two and four year
but two year college
effects,
negative toward SE0A and its
positive about SEOA generally
administrators and faculty were more
their four year counterparts.
and about its impact than were
the
Ewell & Boyer [1988] studied
Not all researchers agree.
including
mandates in five states,
effects of state assessment
to expectation, we found no
Virginia, and reported that "[C]ontrary
to
of institution and response
substantial correlation between type
additional
present study provides
state assessment mandates." The
information on this issue.
products of SEM must be used
Ewell [1985] proposed that the
evidence
difference. Little empirical
if the process is to make a
of
of SECA within institutions
has been published on the effect
Evidence of
higher education.
higher education and external to
student learning and
institution would be improved
use within the
attention toward
administrative and faculty
the redirection of
institution
Evidence of use without the
teaching and learning.
key constituencies and the
would be enhanced credibility among
general public.
studies of community colleges
Geiger [1990] has conducted case
procedural changes in the
He found evidence of
in Virginia.
institutions but no reported
administrative support structure of
in
curriculum, or faculty involvement
changes in student learning,
constituencies
The effect on external
the governance structures.
suggests that state
[1989]
Adelman
is also little
stimii.ed.
or
assurance,
want
legislatures
and
bodies
coordinating
Aper
education.
quality of undergraduate
reassurance, about the
individuals in Virginia and concluded
[1989] surveyed influential
demonstrated
reassurance, not
that the political goal of SEIM was
proof of excellence.
state mandate for
Virginia institutions have been under a
is not nearly
While the Virginia mandate
SEOA for about four years.
[Losak, 1988], SEOA in
example, that of Florida
as rigid as, for
poison" variety -- each
Virginia is clearly of the "name your own
maneuvering room.
institution has some, although only some,
some others,
like
inception the Virginia model,
its
At
involvement in SEOA design,
emphasized the importance of faculty
and
change,
program
interpretation,
and
collection
data
1988; Levin, Lazorack &
institutional governance [e.g., Cress,
1988].
Sears, 1988; Lumsden, 1988; Wyles,
for several years, been
While Virginia institutions have,
to the State Council of
submitting reports on SEOA activities
SCHEV has emphasized data
Higher Education for *virginia [SCHEV],
unclear
involvement. As a result, it
collection more than faculty
4
Levin & Clowes, p.
3
It is clear, however, that
how involved Virginia faculties are.
unless faculties are heavily involved, SEOA is unlikely to prove
either useful or tenacious.
to provide
study was
some
the present
purpose of
The
preliminary indications of how SE0A has developed within Virginia's
institutions of higher education.
In April of 1990 a survey was mailed to the Faculty Senate
and Chief Assessment
Curriculum Committee Chairman,
Chairman,
Officer of 23 community colleges, 15 public senior institutions,
and 28 private senior institutions in Virginia [ergo, N = 66 x 3 =
These three positions were selected because their holders
198].
should be the most knowledgeable about SE0A, if it were operating
questions:
as the Virginia model suggests. The survey asked three
full-time
Approximately what percentage of your
1.
faculty already has been directly involved in development
outcomes
educational
student
of
implementation
and
assessment?
Has the governance structure at your institution
2.
changed at all in anticipation of or as a result of
student educational outcomes assessment?
In your opinion, has assessment become integrated
3.
into, added on to, or had no relationship to normal
academic practices at your institution?
After the smond and third questions, there was space for
comment. Many respondents chose to comment, and what they had to
private
of
representatives
Several
illuminating.
was
say
institutions [including a Dean and a staff member acting at the
called us to inguLre what SEOA was, and
behest of her President]
then assured us that they would investigate forthwith.
As we
The overall written response rate was 39 percent.
expected, the response rate from community colleges was much higher
than tnat of either group of senior institutions [62% vs. 26%,
Also the response rate of faculty senate chairs was lower
27%].
than either curriculum committee chairmen or assessment officers
[26% vs. 44%, 47%]. We expected, and the data sr= to confirm, that
faculty senate chairs were generally not knowledgeable about, or
involved :;n, SEOA.
The joint impact of senior institution status and faculty
senate chair addressee on response rate is overwhelming -- of 15
public
surveys mailed to faculty senate chairs at Virginia's senior
This contrasts with a 52%
institutions, not one was returned.
response rate from community college faculty senate chairs, and an
8241 response rate from community college chief assessment officers.
Clearly this comports well with the notion of a greater [primarily
administrative] interest in SEOA in the community colleges.
Levin & Clowes, p.
4
to
not attributable
response rate
is
differential
This
the largest institution of
institutional size or complexity. By far
major
community college with five
higher education in Virginia is a
and
services,
programs,
of
complex tapestry
campuses and
a
of Higher
[according to the 1990 Chronicle
governance structures
largest enrollment of all
Zducation Almanac it had the 24th
and a relatively well
--
colleges in the U.S. -1988 data]
In
which it communicates freely.
developed SE0A program about
institutions of higher education in
contrast, one of the smallest
Virgin3a is also a community college.
Mean estimates of
instructive.
The responses themselves are
in
public senior institutions, 45%
faculty involvement were 38% in
The
in community colleges.
private senior institutions and 51%
standard deviations in the 30s
variability was unusually high, with
within
tremendous variability
The data demonstrate
and
40$.
This is likely due to lack
institutional and respondent categories.
[consistent with written and oral
SEOA
familiarity with
of
in the wording of the question cannot
comments], although ambiguity
be ruled out.
had been
74 respondents said there
only 15 of
Overall,
indicate that the most
The written comments
structural change.
administrative staff to do SEM.
frequent change was to assign
change in
that much fundamental
Clearly, this does not indicate
that faculty are heavily engaged.
governance has occurred, nor
a
processes,
academic
Regarding integration with normal
Of the public senior,
somewhat less negative picture emerges.
colleges, respectively, 57%,
private senior, and public community
integrated, rather than an add-on.
43%, and 51% believed SECA was
and
when one considers the oral
The rosiness fades, however,
mandated
Sam
a
as
many
by
seen
is
comments.
written
for improving
administrative activity rather than as a process
student outcomes.
The survey data show tnat:
likely to
community college representatives were more
1.
least likely to respond;
respond; faculty senate chairmen were
indicated that any structural change
2. few respondents [20%]
those changes were largely
had occurred secondary to SEM; even
administrative in nature;
regardless of institution type or
3. most written comments,
with SEOA as an administrative
respondent's role, were consistent
not directly raised by the
Even though this issue was
process.
surprised that not a single respondent
survey questions, we are
improvement of any educational
commented that SEM had led to the
other positive outcome other
outcome or that SECA had led to any
than compliance; and
of faculty leadership in SEOA.
4. there is little evidence
f;
Levin & Clowes, p.
5
are
data,
conclusions, based on the survey
Gur tentative
three:
SEGA
is
types,
respondent
and
across college types
1.
administrative process;
primarily perceived as an
they
most vulnerable to SEGA, as
2. community colleges seem
external mandates; and
have been to previous
anything other
yet to prove itself as
3. SEGA in Virginia has
process.
than yet one more reporting
institutional effectiveness, appears
SEGA, like its parent
external mandate.
administrative response to an
most clearly as an
institutional
Welkerls conclusion that
This is consistent with
heading;
efficiency under a new
effectiveness is really management
management efficiency.
SEGA also functions as
we conclude that
demonstrate
own
our
and
data
(1990]
Matlick's
Both
We conclude
institutional type.
differential responses to SEGA by
education institution most
is the higher
the community college
basis for
and see this as the
vulnerable to external pressures,
relied
Ewell & Boyer [1988)
differential institutional responses.
earlier: we feel
conducted their work two years
on interviews and
Matlick's yield
time of this study and
the survey methods and the
point.
better evidence on this
available data
be done, the
Although more research remains to
indicate that there
own data]
[Geiger, 1990; Matlick, 1990; and our
or
academic programs,
student learning,
is little effect upon
administrators are more
Matlick does indicate that
faculty.
concluded that
faculty; Aper [1989]
positive about SEGA than are
than proof of
desired reassurance rather
external constituencies
effectiveness.
colleges the
SEOA affects community
We conclude that while
faculty viewpoint,
change is visible from a
most, and while little
generated
administrative change. Reports are
there has been some
place so that external
are
in
and administrative procedures
for SEGA is
the political agenda
agencies are reassured and
gains
there are any educational
We do not know whether
fulfilled.
institutions
It is possible that some
from the SEGA initiative.
and
opportunity for self-examination
will use the initiative as an
think this is an inevitable or even
improvement; however, we do not
institutions.
likely outcome for most
teaching
model,
behavioral
the
that
observed
have
We
(both as issues
far more prominent
evaluation, and SEOA all became
colleges than in senior
in the community
and as
processes)
relationship between
be an inverse
institutions; there appears to
In
vulnerability to external forces.
institutional status and
instruction have,
and evaluation of
addition, the behavioral model
horizon; it
beneath the higher education
fol the most part, sunk
will follow them.
remains to be seen whether SEGA
7
Levin & Clowes, p.
6
References
Ihg_Rgyigw_gf
Adelman, C. [1989]. The sirens of policy research.
Nigher Educatiort, 12(4), 319-328.
November).
A
study of the development of student
Aper, J. P. [1989,
Paper presented at Association for
assessment policy in Virginia.
Atlanta, Georgia.
the Study of Higher Education,
Clowes, D. A. and Levin, B. H. [1989]. Community, technical, and
Are they
leaving higher education? Journa]_of
junior colleges:
Nigher Education, WQ, 349-355.
[1988]. Toward a collaborative model of academic
Cress,
A.
D.
Illinois.
In
current picture
The
assessment:
S.
in
J.
Reithlingshoefer [Ed.] Developing_assessment partnerships between
cQmmunIty anal, senior solleges: The theory find practice of outcomes
11-25]. Virginia Beach, Virginia: George Mason
assessment [pp.
University.
Ewell, P. T. [1985, November/December]. Assessment: What's it all
about? Cbange, 32-36.
[1988, July/August]. Acting out
T. and Boyer, C. M.
Ewell, P.
Change, 41-47.
state-mandated assessment.
[1990]. Facultv_perceptions of imndated chanae in
Geiger, J.
E.
community colleges in Virginia. Unpublished manuscript, Virginia
Tech, Blacksburg, Virginia.
Levin, B. H., Lazorack, M., and Sears, J. C. [1988]. Development of
a faculty-controlled student outcomes assessment system. In S. J.
Reithlingshoefer [Ed.] Developina assessment_nartnerphiS4 00tWeAA
and seniqr
colleges: The theory and practice of outcomes
COAftinity
46-56]. Virginia Beach, Virginia: George Mason
assessment [pp.
University.
Losak, J. [1988]. Entry- and exit-level testing in Florida and at
Miami-Dade Community College - A brief history with some general
comments on assessment. In S. J. Reithlingshoefer (Ed.] Develogina
assessment partnerships between communiM and senior colleges: The
thegry_anajimagilaiL2Lsjatrames_dasefignent [pp. 57-62]. Virginia
Beach, Virginia: George Mason University.
Designing assessment strategies:
Setting
Lumsden,
[1988].
D.
objectives, selecting instruments, utilizing findings. In S.
J.
Reithlingshoefer [Ed.] Developing_As
community ang senior colleges: The theory and practice of outcomes
63-673. Virginia Beach, Virginia: George Mason
assessment [pp.
S.
Levin & Clowes, p.
7
University.
Matlick, M. A. [1990]. Staff
attitudes toward outcones wisevspent.
(Doctoral dissertation, Virginia
Tech, 1990).
Southern Association of
Colleges and Schools,
Commission
on
Colleges. [1988]. Criteria for
accreditation. Decatur GA: Author.
Welker, W. F.
[1990]. A taxonomy of institutional
effectiveness
literature for public higher education,
colleges, and universities.
[Doctoral dissertation, Virginia
Tech, 1990].
Wyles, B. A. [1988). The status
of
assessment at NVCC: An abstract.
In S. J. Reithlingshoefer [Ed.]
Daisisaninsii-Asaasszsatizartnexabiall
outcomes assessment [pp. 140-143].
Virginia Beach, Virginia: George
Mason University.
Footnotes
1 Blue Ridge Community College,
Weyers Cave VA 24486
2 College of Education, Virginia
Tech, Blacknburg VA 24061
3 Appreciation is expressed
for the comments made by
James R.
Perkins, Blue Ridge Community
College on an earlier draft.
02FE91
4vexi6tAvitee4ocivzieeiNor;w46w
ERIC Clearinghouse for
Junior Col leges
J A N 1 0 1992
,
.
9
4 LNA AartL Joyamiad..".
A-%
&NOM.
4,111k1AWS. VS41,