December 2013 (abridged)

35 Pages·2013·0.68 MB·English

B E
THE
ar xaminer
Volume 82 | Number 4 | December 2013
A publication of the National Conference of Bar Examiners

Articles
6
Predicting the Future Behavior oF Bar aPPlicants
by Peter Ash, M.D.
17
the Mee Marks a Major Milestone
by Judith A. Gundersen
Departments
2
letter FroM the chair
by Margaret Fuller Corneille
4
President’s Page
by Erica Moeser
23
the testing coluMn
Final Musings
by Susan M. Case, Ph.D.
26
news and events
31
litigation uPdate
by Fred P. Parker III and Jessica Glad
B THE E l
ar xaminer etter FroM
c
Editor the hair
Claire Huismann
A Milestone Affords a Look Back
Editorial Advisory Committee
Beverly Tarpley, Chair
Some say it’s a sign of advancing age when you start talking about how things were
Hon. Henry A. Alaniz
“back in the day.” But it’s important sometimes to look back in order to appreciate
Bedford T. Bentley, Jr.
progress that has been made.
Donna E. Blanton
Ann Bowden-Hollis
The retirement of Susan Case, NCBE’s Director of Testing, is an occasion to look
Paul H. Mills
back and marvel at the accomplishments that Susan and her able staff in the Testing
Fred P. Parker III
Hon. Phyllis D. Thompson Department have achieved over Susan’s 12 years with NCBE.
Susan came to the Conference with a wealth of knowledge and experience. Her
Publication Production and Design
undergraduate majors were mathematics and mathematical statistics, and her M.S.
Melanie Hoffman
and Ph.D. degrees are in measurement and evaluation. Prior to joining NCBE, she
Editorial Assistant served for 25 years at the National Board of Medical Examiners with responsibilities
Lisa Palzkill for research and for medical licensure and specialty board examination programs.
During her years at the Conference, Susan showed a unique ability to educate lay-
Publisher
National Conference of Bar Examiners persons in the language and mysteries of psychometrics. Her entertaining and infor-
mative PowerPoint presentations at NCBE’s Annual Bar Admissions Conferences are
NCBE Officers legend. Susan was able to keep a roomful of bar examiners engaged and enlightened
Chair
on a Friday afternoon while she prodded them with the all-important testing concepts
Margaret Fuller Corneille
of sampling, reliability, validity, scaling, and rounding.
President
Susan’s highly readable and relevant Testing Columns in the Bar Examiner exem-
Erica Moeser
plify her skill in educating. She leaves behind a compendium of work that those of us
in bar admissions will return to again and again after her retirement.
Immediate Past Chair
Franklin R. Harrison
Among the many accomplishments that Susan oversaw during her 12 years as
Director of Testing was bringing in-house the test development operations for the
Chair-Elect
Bryan R. Williams MBE, the MEE, and the MPRE. Previously, the questions for each of these examina-
tions had been generated by drafting committees overseen by NCBE and ACT; ACT
Secretary had then handled question editing and test development.
Hon. Thomas J. Bice
What did bringing these three exams in-house mean for Susan and her staff?
NCBE Board of Trustees It meant that all test development work for the four NCBE exams—the MBE, MEE,
Hulett H. Askew MPT, and MPRE—was now handled by NCBE staff members. Questions continued
Hon. Rebecca White Berch
to be generated by NCBE’s drafting committees, but test development steps were all
Robert A. Chong
conducted by NCBE staff. These steps included submitting the questions for outside
Patrick R. Dixon
expert review, editing and pretesting the questions, assessing their performance,
Michele A. Gavagni
Hon. Cynthia L. Martin and tracking equators (reused questions used to scale the current exam to previous
Suzanne K. Richards exams in order to ensure a consistent degree of difficulty) for the MBE and MPRE.
Darryl W. Simpkins NCBE staff also shipped the finished product to the jurisdictions for administration.
And this work was done in addition to overseeing the scoring of the exams and con-
2 The Bar Examiner, December 2013
ducting the scaling work for the MBE for Susan and her staff always made them-
a number of jurisdictions. Needless to say, selves available to assist jurisdictions. They
during Susan’s time as Director of Testing, were always ready to provide expert advice
NCBE’s Testing Department grew signifi- when issues arose such as scoring chal-
cantly—expanding from a staff of one part- lenges or alleged cheating incidents that
time psychometrician to the current Testing needed to be analyzed from a psychometric
and Research Department staff of 15 people. perspective; or when jurisdictions under-
took changes to scoring, scaling, or other
Another significant accomplishment
test-related practices.
during Susan’s tenure was the completion
Susan’s 12 years as NCBE’s Director
of NCBE’s job analysis. Organizations that
of Testing do indeed reflect significant
develop licensure examinations need to
accomplishments. Susan professionalized
periodically assess whether what is tested
the Testing Department and moved the
accurately reflects the knowledge and skills
Conference from one largely driven by volunteers to one
that are needed by newly licensed professionals. In order
that has the resources and expertise to produce high-
to protect the public, the licensing exam must reflect what
quality test instruments and to do so in a way that is cost-
the professional needs to know. Under Susan’s leadership,
efficient.
NCBE conducted a broad-ranging survey of new attorneys
that sought to find out exactly what type of work new From the perspective of the jurisdictions using NCBE’s
lawyers do, as well as what knowledge, skills, and abili- tests, by no means does Susan’s departure mean a loss of
ties these new lawyers believe they need to carry out their the services and expertise that were available under her
work. leadership. Jurisdictions can continue to rely on Testing
Department staff, now under the capable guidance of Mark
The data that was elicited from the job analysis survey
Albanese, former Director of Research, who, as Director of
is extensive. The results of the survey are being studied by
Testing and Research, will now oversee both departments.
NCBE’s Long Range Planning Committee. The informa-
NCBE’s psychometric, testing, and measurement experts
tion gleaned from this job analysis will be used to evaluate
will continue to be available to the jurisdictions for all their
NCBE’s existing exams and to decide if additional content
testing-related needs and to answer all their questions.
areas should be tested or if additional skills should be
The legacy that Susan leaves behind is a rich one—one
assessed. The results will serve for some time as the guide
that serves all of us in the bar examining community. And
for developing and changing exam products in the future.
best of all, what Susan accomplished remains in place to
The job analysis also provides reassurance to the jurisdic-
serve this community in the future.
tions that use NCBE’s tests that what is tested on the bar
exam is in fact consistent with what new lawyers need to Susan, thank you for all you did. We wish you the very
be prepared to do. best in the next chapter of your life.
Susan was also the moving force behind the devel- Best regards to all.
opment of NCBE’s Online Practice Exams, one of the
Sincerely,
most important tools available to examinees to prepare
for the MBE and MPRE. NCBE now offers four Online
Practice Exams for the MBE and one for the MPRE—all
available at low cost to examinees. The exams consist of
recently administered MBE and MPRE questions, and the Margaret Fuller Corneille
answers—correct and incorrect—are accompanied by an
explanation of the law that articulates for the examinee
why each possible answer is either right or wrong.
Letter from the Chair 3
P ’ P
resident s age
by Erica Moeser
The Uniform Bar Examination NCBE has agreed to undertake
has expanded geographi- the task of notifying law schools of
cally! With the recent addi- July and February test results for those
tion of Alaska, the UBE has jurisdictions lacking the resources to
now been adopted in 14 states. Alaska, perform this work internally, because
all 586,412 square miles of it, is a wel- getting this information into the hands
come addition. The state will adminis- of the law schools, and then on to the
ter its first UBE in July 2014. accrediting authorities that are assess-
ing law school performance in accor-
dance with the ABA Standards for
In the September 2013 issue of this
Approval of Law Schools, is so impor-
magazine, I included a chart showing
tant. A number of states have taken us
the extent to which jurisdictions are
up on this offer.
notifying law schools about the success or failure of
their graduates on the bar examination. I am pleased In a related—and encouraging—development,
to share an updated version of the chart on the fol- the Council of the ABA’s Section of Legal Education
lowing page that reflects increased disclosure of and Admissions to the Bar adopted a resolution at its
December meeting in Washington DC calling on the
name-specific information as states have recognized
highest courts of each state and territory to aid in this
the need for disseminating this data.
effort. A copy of the resolution as adopted appears
Disclosure has become a very important con-
on page 40.
sumer protection measure, especially as would-be
Also in the previous issue, I included a chart
law students assess the merits of particular law
using data supplied to NCBE by the ABA Section
schools before plunking down tuition and commit-
that demonstrated the extent to which law school
ting to the student loans that will finance that tuition.
enrollments have fallen off over the past few years.
At a time when law school applications are strikingly
Several law schools notified the Bar Examiner’s edi-
down, I hope that the temptation to fill seats will
tor, Claire Huismann, that there were discrepancies
not induce any law schools to move so low in the
in need of correction. Evidently the data gremlins at
applicant pool that the students who are recruited
the ABA applied their mischief to miscalculate some
are unlikely to pass a licensing examination after
of the numbers. Corrected information appears in
three years of legal education, but none of us can pre- the chart on the following page. (It is worth noting
dict how law school enrollment decisions, entering that the MBE was administered to a record number
predictors such as LSAT scores and undergraduate of candidates this past July—53,706. That number is
GPAs, and applicant demand will play out. sure to fall over the next few years.)
4 The Bar Examiner, December 2013
Finally, Peg Corneille’s tribute to Susan Case (see many better practices have been implemented in
pages 2–3) serves as a nice capstone to Susan’s career jurisdictions because of her talents as an interpreter
at NCBE that culminated with her retirement in early of measurement concepts. Susan was a cheery pres-
November. Susan was a true change agent, and her ence in the NCBE offices. It’s not the same without
contributions to the testing program, and particu- her. We wish her and her wonderful husband Bob
larly to the MBE and the MPRE, were enormously the happiest of retirements.
significant. Susan was also an effective teacher, and
Pass/Fail Disclosure of Bar Exam Results
(Updated chart from September 2013 Bar Examiner)
These jurisdictions These jurisdictions automatically
automatically disclose name-specific pass/ These jurisdictions require
These jurisdictions do not
disclose name-specific fail information to in-state all law schools to request
provide name-specific pass/fail
pass/fail information to the law schools. Out-of-state law name-specific pass/fail
information to all law schools.
law schools from which schools must request the information.
test-takers graduate. information.
CA AR AK AL
CT IN AZ FL*
GA KY CO GU
IL LA DC HI
KS MA DE MP
MD MN IA NH†
ME MS ID NJ‡
MO MT MI PR
NE NC NV PW
NY ND PA SD
OK NM SC TX**
OR OH VI
VA RI WV††
VT TN WY
WA UT
WI
* While Florida furnishes name-specific information automatically to in-state law schools, it provides only aggregate data to out-of-state law schools.
† New Hampshire releases passing information only.
‡ If the applicant executes a waiver, New Jersey will release information to law schools on request.
** Texas will disclose name-specific pass/fail information on request of the law school unless the applicant has requested that the information not be released.
†† If the applicant executes a waiver, West Virginia automatically discloses name-specific pass/fail information to in-state law schools and will release information
to out-of-state law schools on request.
Change in Law School Matriculants from 2010 to 2012
(Corrections to data in the September 2013 Bar Examiner)
Total Matriculants
% Change,
LAW SCHOOL 2010 2011 2012 2010 to 2012
UNIVERSITY OF NEW HAMPSHIRE SCHOOL OF LAW
132 146 74 -43.94%
(FORMERLY FRANKLIN PIERCE LAW CENTER)
NEW MEXICO, UNIVERSITY OF 116 113 114 -1.72%
ST. THOMAS, UNIVERSITY OF (MN) 168 171 143 -14.88%
TOTALS: 52,313 47,187 42,862 -18.07%
President’s Page 5
P F B
redicting the uture ehavior
B a
oF ar PPlicants
by Peter Ash, M.D.
Editor’s Note: This article is based on Dr. Peter Ash’s presentation, “Predicting the Future Behavior of Bar Applicants,”
at the 2013 NCBE Annual Bar Admissions Conference held on April 18–21, 2013, in Boston, Massachusetts.
When considering an applicant who pertaining to each scenario by using response click-
has a history of substance abuse ers that had been distributed to them. This article
or mental health problems, fit- includes some of the audience responses to a few
ness boards are concerned with of the scenarios posed: specifically, those relating to
whether the applicant will be able to function as an alcohol problems, depression, and bipolar disorder.
ethical and competent attorney. The applicant needs The audience members were asked whether they had
to demonstrate that he or she is fit to practice. Such ever served on a fitness board and whether they con-
concerns involve making a prediction about future sented to have their anonymous responses reported.
behavior. In cases of serious past misconduct or The data reported here reflect the responses of the 65
problems, the applicant needs to demonstrate that he audience members who responded affirmatively to
or she has been rehabilitated. those two questions. Percentages given are of those
who responded to the question. Audience members
Such implicit predictions raise a number of
who did not make a choice on a question were not
questions: Can we take the data from studies of the
included in the analyses. The audience members
longitudinal courses of many disorders and apply
were also asked to indicate whether or not their
them to individuals in a meaningful way to predict
jurisdictions allowed conditional admission, as I was
applicants’ future behavior? Can we make reason-
interested to see the breakdown of responses related
able estimates about risk of future difficulties? How
to conditional admission policy; some of that data is
much risk does it take to deny someone certifica-
presented here as well. Before reading the responses
tion? How can we maintain awareness of cognitive
to each scenario, you might pause and consider your
biases that might affect the intuitive judgments that
own answer to get a sense of where you fall in the
are often a part of decision making? How do fitness
distribution of responses.
boards deal with these problems?
In order to gain a sense of how fitness boards
alcohol ProBleMs
approach these issues, I asked audience members
during my presentation at the 2013 NCBE Annual Alcohol-related problems are among the most com-
Bar Admissions Conference to consider several sce- mon problems that fitness boards face. Members of
narios depicted on my presentation slides. I then the audience were asked to respond to the following
asked them to respond anonymously to questions example:
6 The Bar Examiner, December 2013
Example 1: “Social drinker” with DUIs toring (at least two years). Of those who responded
to other questions, 85% responded that they would
John is a 24-year-old third-year law student
require both treatment and urine monitoring. Sixty-
about to graduate. He had two arrests for
three percent thought it more likely than not that
open container violations as an undergradu-
John’s functioning as a lawyer would be significantly
ate and DUI arrests at ages 17 and 22, as well
interfered with at some point in the next 10 years,
as a DUI arrest eight months ago. He refused
and 74% thought it more likely than not that if John
all Breathalyzer tests and pled all charges
did not get treatment, he would be an alcoholic in 5
down to reckless driving. He claims that all
years.
DUIs were the result of heavy drinking at
parties but that he doesn’t drink much dur-
Studies about Alcohol Use
ing the week. He doesn’t see his drinking as
Alcohol Use during College
much of a problem. He says that he now lim-
its himself to two drinks if he has to drive. How do these views match up with what research
says about the course of alcoholism in bar appli-
What would you require?
cants? There has been little research specifically on
law students, but college drinking and the longitu-
Audience Response:
dinal course of drinking after college have received
As can be seen in Figure 1, two-thirds of fitness a good deal of attention. As is well known, college
board members would require John to demonstrate students drink a lot. A major government study
a period of abstinence for at least a year. Not sur- found that about 39% of college students in 2011 had
prisingly, those from states that have conditional been binge drinking in the previous month, numbers
admission would require a longer period of moni- that were four to five percentage points higher than

Figure 1: Applicant with alcohol problems: What fitness board members would require for Example 1 case
Predicting the Future Behavior of Bar Applicants 7
for young people not enrolled in college. This level Exploring Predictors
of drinking was slightly reduced (by about 5%) from
There has been considerable research into the ques-
rates in 2002. Sixty-one percent drank some, and 13%
tion of what predicts the course of alcoholism, and
were heavy drinkers. In addition, 21% of 16–20–year-
the bottom line is that there are no simple answers.
olds reported that they had driven in the past year
The best models utilize numerous variables, such
while under the influence of alcohol or illicit drugs.1
as negative life events, motivation for change, cop-
ing resources, craving experiences, mood status,
Alcohol Use after College
participation in treatment, amount of drinking at
The good news is that much of this drinking tapers the study’s outset, level, sex, age, moral beliefs,
off after college. The rates of alcohol dependence for and fear of loss of control (fear of becoming rude
those age 25–29 are about half of the rates for those or obnoxious, becoming alcoholic, or getting into
age 18–20,2 and rates of alcohol dependence continue trouble).6 All of these variables have some predictive
to fall into middle age. value (interestingly, participation in and duration of
treatment appear more significant than the type of
Who continues to drink? For those followed
treatment chosen), but with so many variables and
from age 21 to age 31, the strongest predictor of
so many different paths to sobriety, the models are
drinking at age 31 (accounting for 75% of the predic-
complex. While some models achieve statistical sig-
tive power) was the amount of alcohol consumption
nificance in determining differences between groups
10 years earlier.3 Also clear from longitudinal stud-
of heavy drinkers who differ on the variables listed
ies is that for many drinkers, the amount of drinking
above, the models have very limited power in terms
varies considerably over time. Over half of alcohol
of explaining individual outcomes.
abusers eventually have a period of no serious
impairment for at least one year, and for those who
The Difficulty with Attempting Individual Predictions
have such a period of sustained recovery, the major-
The statistics I’ve discussed are group statistics. With
ity have done so with the help of treatment, either
what confidence can we measure variables in one
through Alcoholics Anonymous or with a mental
health treatment provider.4 person and make an accurate prediction? What we
would like is a model where we could enter a variety
However, even those with a year of sobriety of measurements about an applicant and obtain an
remain at risk for relapse. George Vaillant, Professor accurate prediction of how likely he or she is to be
of Psychiatry at Harvard Medical School, has con- a heavy drinker several years later. The state of the
ducted the longest follow-up study (60 years) involv- art is such that no accurate predictive model for an
ing alcohol consumption and found that “return to individual currently exists. Lack of treatment and
controlled drinking rarely persisted for much more lack of ability to reduce drinking certainly indicate
than a decade without relapse or evolution into significant risk in the short term, and for someone
abstinence.”5 Most studies do not have extended with alcohol dependence, return to controlled drink-
follow-up for more than a year or two, and Vaillant’s ing is likely not to work in the long term, but even
work suggests that a return to controlled drinking, those predictive statements come with considerable
as reported in short-term studies, is often a mirage. uncertainty.
8 The Bar Examiner, December 2013
Findings of the Levin et al. Study Testing Bar Discipline of an adverse outcome is low, say 2.5%, then if a
Predictability particular characteristic, say heavy drinking, qua-
druples the risk, it means that only 10% of the heavy
The hypothesis that variables measured at the time
drinkers would go on to have the adverse outcome.
of application to take the bar can predict later bar
For a fitness board to deny certification to everyone
discipline was tested in a recent study by Levin et
with a 10% risk would mean that 90% of the people
al.7 Using data collected from a large sample of appli-
denied (the false positives) would not be expected
cants to take the bar examination, they compared
to have later problems. Put another way, if a fitness
those attorneys who subsequently (up to 20 years
board denied certification to 100 people who had
later) received bar discipline with those who did not.
such a risk factor, it would prevent 10 people from
They found that rates of substance abuse at the practicing law who would be expected to get into
time of application were higher among disciplined trouble (the true positives), but it would also deny
lawyers than among never-disciplined lawyers the other 90 people who would not otherwise have
(1.38% vs. .92%), but utilizing variables measured problems (the false positives). The difficulty is in
at time of application, including substance abuse, telling if a person is a true positive or a false posi-
did not predict with much accuracy who would be tive, and group statistics don’t help one to do that.
disciplined. When combined with other factors, sub- For a base rate outcome of 2.5%, in order to say that
stance abuse did not even stand out as a significant a person is more likely than not to have problems
predictor of later discipline. later, it is mathematically required that a person
with that characteristic be more than 20 times (50%
This difficulty in finding accurate predictors
÷ 2.5%) as likely to have the adverse outcome than
was not limited to alcohol problems in the Levin et
someone without that characteristic. The Levin et
al. study. While they found that a number of factors
al. study and a similar study of physician discipline8
doubled the risk of later discipline (including male
found variables that at most doubled the risk of later
sex, graduation from a lower-ranked law school,
professional discipline.
criminal conviction, driver’s license suspension,
and psychological disorder), these variables, even in
Relating Audience Responses to the Data
combination, did not provide a predictive model of
much accuracy. The authors attribute much of the How do the audience responses to the example of
difficulty in predicting to the low base rate of disci- John with his three DUIs fit with these findings? The
pline. The base rate in this context is the percentage majority’s requirement for a period of at least a year
of all attorneys who received bar discipline—2.5% in of mandated sobriety is likely to weed out those who
their sample. Of course, bar discipline is an imper- simply cannot stop drinking, and it may constitute
fect measure of problematic conduct; the actual rate a type of treatment intervention, which lowers risk.
of problematic behavior is no doubt considerably The data suggest that a year of sobriety by itself has
higher but not as easy to measure. very limited value in predicting the future course,
as alcoholism is a relapsing illness and it is very
Why Low Base Rates of Discipline Complicate
difficult to predict who will relapse. The pessimism
Predictions
exhibited by the 63% of bar fitness board members
Low base rates of an outcome create particular dif- who thought that John’s functioning as a lawyer
ficulties in making accurate predictions. If the risk would more likely than not be seriously impaired in
Predicting the Future Behavior of Bar Applicants 9
the next 10 years does not appear to be warranted by has little insight into her condition, has
the available data, although the data are not conclu- moderate depressive symptoms, but has
sive on this point given that while we know the base functioned reasonably well in her job as a
rate for formal attorney discipline, we do not know paralegal for the past year.
the base rate for impaired functioning by alcohol.
What would you require?
dePression
Audience Response:
Depression is probably the most common serious
mental health issue in the population. Members of As can be seen in Figure 2, there is a U-shaped
the audience were asked to respond to the following curve, with a significant proportion of fitness board
example: members prepared to certify now or after a rela-
tively short period, possibly because the example
Example 2: Past suicide attempts
implies that Clare is functioning well presently and
Clare is a 26-year-old applicant with a his- that her last admission was two years ago. Fitness
tory of three psychiatric admissions follow- board members from states that have conditional
ing suicide attempts between the ages of admission were more likely to monitor the example
15 and 24. She is not currently in treatment applicant and to do so for a longer period. However,
because, she says, she “grew up.” She had when the audience was shown a follow-up question
an independent medical examination, which asking what they would do if Clare made a further
found that she has never been psychotic, suicide attempt three months later, 44% said that

Figure 2: Applicant with depression: What fitness board members would require for Example 2 case
10 The Bar Examiner, December 2013