ERIC EJ822741: Accountability "Light": Our Version Is Going the Way of the Dollar vs. the Euro

2008·0.42 MB·English

by  ERIC

CLIFFORD ADELMAN
Accountability
“Light”
Our Version Is
C
I Going the Way of the
P
O
Dollar vs. the Euro
T

D
E
R
U
T
A
E ITWASJULY14, 2006, when the Spellings Commission on the
F
Future of Higher Education posted the second draft of its final re-
port on the Web for all to see. It was the first of
the commission’s drafts in which the alleged illit-
eracy of U.S. college graduates was placed in bold space along
The accountability with all the other flagellations of higher education in which the
commission delighted. The text of the Web posting, though, was
framework that has
an unhappy validation of the complaint: the cases of violation of
been developed
subject-verb agreement leaped off the electronic pages, along with
through the Bologna paragraphs that can be kindly described as strings of non-sequiturs.
Process is worth The commission’s draft would not be awarded a C- in English
Composition on a generous grading day. Yet no one noticed the
learning from and
irony. After all, this was a distinguished panel. Higher education
thinking deeply about
had not seen a federal undertaking such as this since 1984, and it
was getting visible attention in the general press for topics other
than college costs.
But it would not have been hard to notice, in both the Web-
posted drafts and the final report of the commission, that the only
references to higher education outside the United States recited
ill-informed data putatively demonstrating how far we had “fallen
behind” other nations.1 Nobody on the commission or its staff
had bothered to examine the ways other countries are dealing
with issues we also face—access, degree completion, and, most of
all, accountability. In its insular and chauvinistic way, this major
pronouncement on, and recommendation of remedies for, the state
of U.S. higher education pretended, without looking, that the world
is learning more, and we are not—a conclusion designed more to fit
our appetite for bad news and self-degradation than to enlighten.
CLIFFORD ADELMAN has been senior associate at the Institute for Higher
Education Policy since the fall of 2006. Prior to that, he served for
twenty-seven years as senior research analyst at the U.S. Department of
Education, and ten years in both teaching and academic administration.
6 LIBERAL EDUCATION FALL 2008
Franklin College
C University of Now, if the rest of the world is learning more,
I Provence, would that be surprising? And in a global econ-
P
Aix–Marseille omy, would it really be so disappointing? Cul-
O
T ture and language ensure that the world is not
flat, but in terms of knowledge it is, and the
D
E world’s knowledge content, like the oceans, is
R rising. To the extent that the world learns more,
U
we all benefit, and we cannot pretend that the
T
A United States holds a monopoly on knowledge.
E More seriously, because credentials awarded are
F
used as a proxy for learning in international—as
well as domestic—comparisons, nobody knows
for sure whether students in other countries are,
in fact, learning more. But nowhere in recent
years have academic leaders, faculty, and stu-
dents themselves wrestled more with the knots
of credentials and learning than in the old na-
tions of Europe. Through the largest recon-
struction of higher education ever undertaken
anywhere—known as the Bologna Process—
they have come up with a convincing and cred-
ible scaffolding of accountability. When one
considers what the Europeans have done—and
where they have more to do—one realizes that
we in the United States have a long way to go
in understanding what “accountability” means.
The Bologna Process
The Bologna Process is named for the city where,
in 1999, the education ministers of twenty-
nine countries first reached a meeting of the
minds on the future of higher education in
Europe. It is an inevitable outgrowth of the
process of European integration that began in
1950. By the 1990s, and certainly after the
Maastricht Treaty of 1992, Europe began to re-
semble a quasi-federal arrangement. Although
there were few economic borders, the common
European workforce was, ironically, stuck
within political borders because individual
countries did not yet fully recognize—or even
understand—their neighbors’ education cre-
dentials. In order to allow for the recognition
of higher education credentials across borders
and, thus, to provide mobility for the advanced
knowledge workforce, some convergence of ed-
ucational practices and standards was needed.
Bologna became a force for converting edu-
cation systems into similar forms and processes.
Its discourse is a type of technology transfer that
brings nations from different platforms of edu-
cational development to the point of embracing
compatible paradigms. In other words, they
wind up singing in the same key—thoughnot
8 LIBERAL EDUCATION FALL 2008
It is worthwhile to
the same tune. With its simul- consider whether and fees, they need only to
C
taneousrespect for distinctive some elements might make public their graduation I
P
national traditions and institu- rates, demographic mix, job
usefully be adapted to O
tional autonomy, the Bologna placement rates, and some T
higher education
Processbecame doubly attrac- data from the National Survey
D
tive as theonly game in town. in the United States of Student Engagement on E
In 1999 and subsequently— what their students like about R
U
forty-six countriesare now in- opportunities and services.
T
volved—the ministers agreed to an action They may perhaps also throw in a test or two A
agenda for dissolving educational borders in to show that a sample of their students know E
F
the same way that economic borders had been how to write or solve a problem or that a ran-
dissolved. The agreement was an inevitable dom set of one hundred student volunteers
consequence of reformsthat had beenstirring produced an X-standard deviation improve-
across European education during the 1990s. ment between entrance and graduation on a
The feeling was that, having lost their way test of “critical thinking” (which improve-
and their world leadership, ancient systemsof ment, of course, is attributed entirely to the
higher education needed a kick-start. In 1998, institution2). But this is show, not substance.
the ministers of the four largest countriesin Eu- Wish lists for student learning, flowery goals
rope—France, Germany,Italy, and the United statements, and agendas of commitment for
Kingdom—met at the Sorbonne in Paris and the twenty-first century are sometimes offered
planted the seeds from which the European in parallel rubrics and posed as accountability
Higher Education Area has grown. The statements. But none of these efforts explains
Bologna ministers optimistically set 2010 as what credentials represent or what students
the date by which the transformations they must do to earn them. There are no public ref-
imagined were to be realized. But inevitable erence points or performance criteria. Stu-
inertia andresistance at the institutional level, dents—our constituents who ought to be our
new provisions, and additional partners have partners as well—played no role in fashioning
rendered the 2010 completion marker a mirage; these efforts. And given how divorced these
2020 is more likely. various approaches are from the actual work
The Bologna Process is a huge undertaking. of most faculty and their students, their likely
Its action lines include revising degree cycles, impact on student learning is limited indeed.
establishing degree qualifications frameworks, So what are the Europeans doing differently?
overhauling the credit system, increasing access They have essentially created an accountability
via more flexible paths into and through higher loop that is executed by national systems and
education, validating student attainment, es- institutions. Although still a work in progress,
tablishing a quality assurance system, and in- the European accountability loop begins with
creasing the attractiveness of the European degree qualifications frameworks and circles
Higher Education Area for foreign students. back with a document that serves as a public
Some four thousand institutions, which enroll warranty, attesting to the institution’s judg-
sixteen million students, are now participating. ment and the student’s achievement. In be-
Some of what has been accomplished has al- tween are credential qualifications frameworks
ready been adapted in Latin America, North at the disciplinary level and a student-centered
Africa, and Australia. Indeed, the Bologna credit system to which a growing number of
model is well on the way to becoming the dom- institutions have added levels of challenge.
inant global paradigm for higher education by Countries outside of Europe have already
roughly 2025. We cannot afford to ignore it. adapted some of these elements for use within
their national systems or institutions, and it is
Real accountability worthwhile to consider whether some elements
The Spellings Commission report and the re- might usefully be adapted to higher education
sponse to it from within the U.S. higher edu- in the United States.
cation community give the impression that
accountability is a very simple matter. In order Qualifications frameworks
for colleges and universities to be “accountable” Within the context of the Bologna Process, a
to those who subsidize them or pay their tuition qualifications framework is a statement of the
FALL 2008 LIBERAL EDUCATION 9
learning outcomes and competencies that a One might expect each national higher
C
I student must demonstrate in order to be education system to develop its own compati-
P
awarded a degree at a specific level. It is not ble version of the Qualifications Framework
O
T simply a statement of objectives or goals; it is for the European Higher Education Area—
nota wish list. Rather, it is a set of performance adding detail, taking into account the pecu-
D
E criteria. When an institution of higher educa- liar varieties of its own institutions as well as
R tion is governed by a qualifications framework, their historical missions and commitments,
U
it must “demonstrate” that its studentshave and perhaps including “intermediate” qualifi-
T
A “demonstrated”—allof its students, not merely cations between the three degrees. In prac-
E a hundred volunteers who take a standardized tice, however, that’s not how the national
F
test. While a qualifications framework does qualifications frameworks have developed.
not dictate howthat demonstration takes Creating consensus on a national framework
place or the nature or form of assessments is a time-consuming challenge, and to date,
employed, it does provide learning outcome only seven of the forty-six participating coun-
constructs within which the demonstration tries have completed the task. Among them
is conducted. are five highly distinct models that illustrate
A second key characteristic of a qualifica- the way the Bologna countries can achieve
tions framework is that it clearly indicates the “convergence” with variation. The Irish and
criteria for each level of credential offered. Scottish framework is comprehensive, from
The language of the framework accomplishes kindergarten through the doctoral level,
this through a “ratcheting up” of learning while the German framework offers a more
benchmarks—that is, by increasing the level parsimonious articulation of how university
of challenge at each rung on the degree lad- students must demonstrate knowledge through
der. This “ratchet principle” pervades all of what are called instrumental competencies,
the content challenge and performance state- systemic competencies, and communicative
ments of Bologna, from individual courses to competencies. The Dutch framework refer-
degrees, and has penetrated the credit system ences labor market positions and tasks, and
as well. This principle embodies content and the Swedish framework reaches into nineteen
performance standards, and it functions as an specific applied disciplines that lead to licensure
engine of accountability. occupations (e.g., nursing). The French frame-
There are three levels and types of qualifica- work requires every degree program in the
tions frameworks under Bologna: transnational, country to undergo a central registry review,
national, and disciplinary/field. The trans- and the program dossier is made public via
national Framework of Qualifications of the the Internet.
European Higher Education Area provides a
wide-angle set of generic markers that clearly Tuning and benchmarking
distinguish among the “short-cycle” degree The disciplinary qualifications frameworks are
(the equivalent of our associate’s degree), the developed through what is called the “tuning”
bachelor’s degree, and the master’s degree. The process. “Tuning” is a methodology for pro-
distinctions are drawn not in terms of a num- ducing “reference points” that faculty can use
ber of credits or a minimum grade point aver- in developing statements of learning outcomes,
age, but rather in terms of student learning levels of learning, and desired competencies
outcomesacross five bands of requirements: in the disciplines. The purpose is to ensure
knowledge and understanding; application of that these statements are both transparent
knowledge and understanding; fluency in the and comparable. This does not mean that the
use of increasingly complex data and informa- content is standardized, however. The crite-
tion; breadth, depth, and range of audiences rion-referenced competency statements that
for communication; and degree of autonomy result from the tuning process are not straight-
gained for subsequent learning. The language jackets. To reprise the music metaphor: every-
used to describe these requirements “ratchets body winds up with the same music staves,
up” the level of challenge as one moves up range of time signatures, tempo commands,
the degree ladder. This framework has been and system of notation. Then, all programs in
accepted by all forty-six countries participating the same discipline sing in the same key—en-
in the Bologna Process. gineering in A-minor, business in B-flat—but
10 LIBERAL EDUCATION FALL 2008
The importance
they don’t necessarily sing the of discipline-specific our students enter a major—
C
same tune. The first phase of content has been a and when they graduate— I
P
the tuning project involved they deserve to know what they
consistent theme O
nine fields in 138 institutions are in for, and, later, what they T
across sixteen countries. In of the Bologna Process have accomplished. Students
D
the second phase, sixteen are not likely to get this clarity E
other fields joined the model. from the institutional gradua- R
U
Tuning is the largest Bologna“export” to date: tion rates, indicators of student engagement,
T
twelve disciplines in 182 universities in eigh- “value-added” metrics of “critical thinking,” A
teen Latin American countrieshave adopted and generalized curricular goals one finds in so E
F
the methodology. many “accountability” pronouncements in
However attractive “tuning” may be, it is the United States. Our students deserve better.
not always executed to match its purpose.
Faculty members have difficulty writing crite- Euro-Credits
rion-referenced learning outcome statements, While the European Credit Transfer System
and that’s something the Europeans—and we (ECTS) is built on a fundamentally different
in the United States—need to work on. Disci- assumption from that used for U.S. higher ed-
pline-based benchmarking, a strong suit of the ucation credits, if it plays out in its ideal form,
United Kingdom’s Quality Assurance Agency it will go beyond functioning as a measure of
(QAA), is an especially prominent alternative. accumulation and become another account-
Like tuning, benchmarking provides reference ability tool.
points and boundaries for designing, modify- In the United States, credit assignments
ing, and evaluating the presentation of a disci- are based on faculty contact hours, with the
pline within an institutionor a group of assumption that, in relation to each faculty
similar institutions. The QAAbenchmarking contact hour, the student engages in other
statements for degrees in accounting, for ex- types of learning activities. In contrast, ECTS
ample, clearly describe the subject-specific uses the studentas the primary reference point,
knowledge, skills, and cognitive operations asks how many hours the average student must
that the graduate will have “demonstrated” at spend to accomplish the various tasks in a
a level crossing a threshold standard. While course module, and converts the total to credits.
institutions choose their own forms of assess- The conversion formula is based on the length
ment and set their own thresholds, a student of the academic year, the total number of hours
who does not “demonstrate” does not earn available for study during that year, and a refer-
a degree. ence marker of sixty credits per academic year.
So what can we learn from the European The conversion formula ranges from twenty-
experience with qualifications frameworks, five to thirty workload hours per credit.
tuning, and benchmarking? Accountability The virtues and drawbacks of such a system
begins with the establishment of public defin- are obvious. When faculty really have to
itions of degrees and criterion-referenced think about what they are asking students to
statements of performance. Once such defini- do, there is strong potential for curricular
tions are established, the credential-awarding change. Redundancies, gaps, and opportuni-
institution can say, “this is what this student ties for cyber-efficiencies can be identified,
did, and this is what the degree represents.” and appropriate changes in coverage and de-
The importance of discipline-specific con- livery can be made. The Bologna literature
tent has been a consistent theme of the describes such reforms in fields ranging from
Bologna Process. After all, students go to col- music to chemistry. The obvious drawback can
lege to earn a degree in anthropology, me- be simply phrased: when you have a formula,
chanical engineering, or nursing; they go to it becomes the default. Water finds the easiest
community colleges to earn a degree in med- way to flow downhill, and both academic
ical technology or commercial art. When you administrators and faculty become much too
ask them what they are studying, they will an- mechanical and downright sloppy in assigning
swer by telling you about their discipline or credits to course modules.
field. And, of course, faculties the world over How does one connect student workload
are also organized by discipline or field. When to learning outcomes, to the principle of the
FALL 2008 LIBERAL EDUCATION 11
ratcheting up of challenge and, hence, to structures of challenge, what is left in this very
C
I qualifications frameworks and the structure of different scaffolding of accountability? What
P
accountability? We don’t worry about such evidence of learning and attainment does the
O
T things in the United States. We grant three graduate carry forward into the world, and
credits for Introduction to Sports and three how is that evidence communicated for allto
D
E credits for Econometrics or assign nine credits see? The piece of paper we call a diplomadoes
R for a Recreation Practicum and four credits for not say much, and our European colleagues
U
Neuropsychology—and brush such obvious have added something very important to it.
T
A dissonances under the rug. Nobody—not the As the final element in the Bologna ac-
E Spellings Commission or the various “ac- countability loop, the Diploma Supplement
F
countability”-oriented responses of the U.S. serves as a kind of warranty, providing evidence
higher education community—has evidenced of the graduate’s learning and attainment. In
any desire to acknowledge, let alone repair, addition to including a transcript and indicat-
these obvious holes in the academic integrity ing superior performance (i.e., honors), the
of our credit system. In the United States, Diploma Supplement is intended to convey
credits live in the office of the vice president information about
for finance, not the vice president for academic •the national system in which the degree
affairs; the student is incidental to it all. If we was awarded and the position of the degree
care about accountability for student learning, within that system’s credentialing hierarchy;
and if we are serious about matching practice •the status, type, and accreditation of the
to rhetoric, then we may need a redesign. If institution awarding the degree;
ECTSplays out in ideal form even among a •the purpose and function of the credential;
plurality of Bologna participants, it will show •the student’s major field of study as well as
us the way. the duration and the entry requirements of
Given different modes of student work in the the program in which the credential was
disciplines, our European colleagues have gone granted;
about the task of linking workload to learning •discipline-level qualifications and degree
outcomes with alternative proxies. In the most requirements, including internships, theses,
intriguing of these approaches, that taken by and final projects;
the United Kingdom and Scotland, credits are •modes of study, including enrollment inten-
placed within degrees of challenge. That link— sity and distance learning.4
between the measure of estimated student Even if they included only the information
time-on-tasks and the level of demand inherent listed above, Diploma Supplements would go
in those tasks—creates a “credit level” defined some way toward providing public assurance
as “an indicator of the relative demand, com- about higher education credentials. Some
plexity and depth of learning and of learner serious revisions are needed, however. The
autonomy.”3There are nine such credit levels, absence of discipline-level qualifications state-
and each carries a generic description that is ments, for example, is a very unhappy omis-
independent of discipline but applicable to all sion in light of the qualifications frameworks
disciplines. Every course is tagged with a credit core of Bologna. And the intended warrantee,
level, and the number of credits awarded is the student, plays too minor a role in the cur-
treated as a separate issue.Once levels such as rent version of the Diploma Supplement for
these are established, degree qualifications can the loop to close.
be set in terms of minimums—40 percent of In addition to providing a public warranty
credits at level six, for example, and 65 percent for our students, the creation of a U.S. version
of credits at levels five and six. If all colleges of the Diploma Supplement would put institu-
and universities in the United States added tions of higher education on the public record
such indicators of the challenge of content to in terms of their standards for degree qualifica-
credits, quarrels about credit transfer would tions, and it would hold them to consistency.
diminish considerably. Setting aside the transcript as a separate docu-
ment (as it should be), what should our version
Diploma Supplements of the Diploma Supplement contain in addi-
After qualification frameworks, tuning, and tion to the bullets above covering the institu-
an ideal credit system marked with parallel tion, the degree and its function, modes of
12 LIBERAL EDUCATION FALL 2008
study, and “compressed signals” of superior aca- about, and I urge academics in the United
C
demic performance? I advocate the following: States to take it very seriously—before the I
P
•Not-so-standard boilerplate indicating all “dollar” of their enterprise loses further value,
O
other institutions attended by the student no matter how many degrees they award. ■ T
from which credits were accepted as well as
D
the percentage of the student’s credits that To respond to this article, e-mail [email protected], E
were earned at the institution awarding the with the author’s name on the subject line. R
U
degree
T
•A tuning-type qualifications statement for A
the student’s major and information about NOTES E
any required internships, theses, final pro- 1.Comparisons among national systems rely princi- F
jects, portfolios, or comprehensive exami- pally on the methodology of the Organization for
Economic Co-operation and Development (OECD)
nations
and its highly visible vehicle, Education at a Glance.
•The title and a short description of the stu-
Yet OECDitself has for some time been unhappy
dent’s thesis or final degree-qualifying with its higher education indicators and, in the re-
project, if applicable cently released 2008 edition of Education at a
•Any external certification examinations Glance, has made the first of a series of changes in
data reporting: an indicator based on cohort gradu-
passed or licenses granted to the student
ation rates for first-time, full-time students in bach-
(although the institution is not the award-
elor’s degree-granting institutions has been added.
ing body in these cases, the institution OECD’s traditional indicator for postsecondary at-
certifies that it has recognized and recorded tainment relies on population ratios—i.e., from
them) census data or labor force surveys in each country,
the number of people holding postsecondary degrees
•A maximum of two noteworthy and docu-
in relation to the total working-age population.
mented services performed by the student for
Variations on this indicator also use population
the institution or its surrounding community ratios for specific age brackets (e.g., twenty-five to
•Student research, creative, or service par- thirty-four). The problems with this approach are
ticipation, if applicable (field, title of pro- numerous, beginning with different census
methodologies and different definitions of degrees.
ject, and faculty sponsor)
This article is not the occasion for elaboration. For
•Documented proficiency in languages other
an overview, I recommend Jane Wellman’sApples
than English, including method of docu- and Oranges in the Flat World: A Layperson’s Guide
mentation to International Comparisons of Postsecondary Educa-
To be effective and credible, these markers on tion(American Council on Education, 2007). I
also recommend reading the technical notes in Ed-
a Diploma Supplement should be limited,
ucation at a Glance—particularly for the new indi-
based on unobtrusive institutional records of
cator—as the notes demonstrate national variances
the student’s activities, concentrated on in definitions, inclusions, and exclusions.
achievements related to the degree awarded, 2.Not considered are a host of potential intervening
and verifiable and validated by whoever signs variables such as whether the student volunteers
were transfers-in or attended other schools as un-
the document on behalf of the institution.
dergraduates, whether they were parents and/or
Otherwise, they are properly part of a resume.
veterans, whether they held jobs that provided
Such a Diploma Supplement would close learning challenges, whether they practiced debate
the accountability loop. To repeat: it is both a in Bible or Talmud study in religious institutions,
public warranty and a private assurance of the and whether their significant others spoke languages
other than English.
meaning of the degree, of the standards for
3.Credit and HE Qualifications (Joint Credit Bodies
awarding it, and of what the student did to earn
for England, Wales, and Northern Ireland, 2001).
it. Thus the loop returns to the public defini- For text of a sample of four of these credit levels,
tion of degrees and public criterion-referenced see The Bologna Club: What U.S. Higher Education
statements of performance, with real students Can Learn from a Decade of European Reconstruction
(Institute for Higher Education Policy, 2008), 60.
carrying the evidence of having met both. This
4.The official “Outline Structure for the Diploma
would be a far more convincing enactment of
Supplement” is available online at www.ec.europa.eu/
accountability than what we have seen in the education/policies/rec_qual/recognition/ds_en.pdf.
United States over the past two years.
The accountability framework that has
been developed through the Bologna Process
is worth learning from and thinking deeply
FALL 2008 LIBERAL EDUCATION 13