ERIC ED371580: Simply Defining: Constructing a Dictionary of Language Testing.

15 Pages·1993·0.29 MB·English

by  ERIC

DOCUMENT RESUME
FL 022 051
ED 371 580
Davies, Alan
AUTHOR
Simply Defining: Constructing a Dictionary of
TITLE
Language Testing.
PUB DATE
93
15p.; In: Tickoo, M. L., Ed. Simplification: Theory
NOTE
and Application. Anthology Series 31; see FL 022
043.
Research/Technical (143)
PUB TYPE
Reports
MF01/PC01 Plus Postage.
EDRS PRICE
Definitions; *Dictionaries; Educational Media;
DESCRIPTORS
*Encyclopedias; Foreign Countries; *Graduate
Students; Higher Education; *Language Attitudes;
Language Research; *Language Tests; Language Usage;
*Readability; Testing
Simplification (Language); University of Melbourne
IDENTIFIERS
(Australia)
ABSTRACT
This paper examines the nature of dictionaries and
encyclopedias, focusing on some of the preparation that has gone into
the construction of a dictionary of language testing at the
University of Melbourne (Australia). It discusses the purpose of such
dictionaries, the nature and size of dictionary/encyclopedia entries,
and the readability of entries. It also reports on an experiment to
determine the proper length and difficulty level of possible entries.
Twenty-one M.A. students were asked to read three sets of entries and
comment on their length, difficulty, and lexical density. It was
found that there was substantial agreement among the students as to
whether several entries were too .long, too short, or about right, and
if they were difficult, easy, or about right. An appendix contains
copies of the possible dictionary entries. (MDM)
***********************************************************************
*
*
Reproductions supplied by EDRS are the best that can be made
*
*
from the original document.
***********************************************************************
0
ct)
it)
;Z.
C.,
LAJ
SIMPLY DEFINING
CONSTRUCTING A DICTIONARY'
OF
LANGUAGE TESTING
Alan Davies
U.S. DEPARTMENT Of EDUCAT)ON
-PERMISSION TO REPRODUCE THIS
Othce of Eclucahonat Research and Improvement
MATERIAL HAS BEEN GRANTED BY
E DUCA TIONA L RESOURCES INFORMATION
CENTER (ERIC)
\kit, nc
11.
aS
.)rhis document has been reproduced
received from the PorgOn or onganitalion
originating it
V,Lern
O Minor Changes have been made to improve
re0naduCtcOn gullibly
Pont s of view or opinions stated fri this docu-
TO THE EDUCATIONAL RESOURCES
ment do not necesssnly represent official
OEM position or policy
INFORMATION CENTER (ERIC)."
SIMPLY DEFINING
OF
CONSTRUCTING A DICTIONARY
LANGUAGE TESTING
Man Davies
it comprehends the full extent of
compile a Dictionary undertakes that which, if
q-le that undertakes to
knows himself unable to perform'
kis design, he
(Johnson 1773)
c.unpound,
the term 'simple' thus: 'Every
C S Lewis (1960) comments on
ingredients
into simple ingredients,
hope, can in principle be resolved
or so we
is a compound, so these
homogeneous. And as the compound
which arc internally
plain is the reverse of
simples.' (166). 'What is simple or
ultimate ingredients are
argument
is hard to learn and a complicated
complicated. A complicated process
The idea that it is within
Therefore simple comes to mean 'easy'.
hard to follow.
'unskilled') may perhaps have
those who are simple (in the sense
the capacity of
of the word as a
(174). 'I describe the final state
helped this development.'
precise sense but a
effectively remains is not this or that
semantic sediment. What
disanningness.' (179).
general appealingness or
there is no objective judgement as
Disarming indeed! Does this mean that
appealingness, a kind of
difficult, that it is all a question of
to what is simple or
lexicon?
political correctness (P.C.) of the
of
has been a great deal of investigation
As far as texts are concerned there
fmdings of
Jeanne Chall (1984) summarises the
what makes for ease of readability.
this work, thus:
research in
read and comprehend? The
'What makes text easy or hard to
such
hundred factors related to difficulty -
readability has uncovered over one
content,
concepts, text organization,
factors as vocabulary, sentences, ideas,
Of these factors, the two found
abstractness, appeal, format and illustrations.
comprehensibility are vocabulary
consistently to be most strongly associated with
in
these two factors are included
difficulty and sentence length. Various forms of
the two is
formulas. The strongest factor of
most of the currently used readability
of unfamiliar words, hard words,
vocabulary diffIculty - measured either by a count
syllables, or words of 7 letters or
words of low frequency, words of three or more
101
3
ITAILABLE
BEST COM"
factor is used
Once a vocabulary
highly interrelated.
word measures are
length is the
more. All
Average sentence
the prediction.
another adds little to
difficulty in readability
in a formula,
used measure of
second most widely
difficulty, and
second strongest and
of syntactic
other measures
highly related to
It is also
formulas. It is very
in a formula.
is usually used
sentence factor
therefore only one
and a sentence
difficulty. A vocabulary
with vocabulary
degree
substantially associated
text to a high
difficulty of written
the comprehension
reading
0.7 to 0.9 with
factor together predict
run about
multiple correlations
of accuracy. The
tests.' (237-8).
multiple-choice or doze
comprehension in
contribute to
other factors that
deny that there are
Chall does not of course
she
will focus on the two
of this paper we
but for the purposes
will not here use any
text readability
length, although we
and sentence
details, vocabulary
readability formulas.
for a dictionary (or
of information
with the preparation
Our concern is
for the distinction).
see below
encyclopedic dictionary,
specifically for an
the construction of a
more
that has gone into
of the preraration
Testing
The paper reports some
NLL1A Language
1992) at the
Testing (Davies
Dictionary of Language
Melbourne.
Centre, University of
preparation of
the institutionalised
available to others,
Making information
of simplification.
is necessarily a process
heart of all pedagogy,
delivery of
data to inform, the
the pedagogic
define simplification as
we may
existing in nature
Or to put it another way
possible difference
that is an issue of
information. Not simple,
that there is no
than a Y. Observe
X is more difficult
whereby for example an
continuum in nature
the simple-difficult
in other words
limitation here as to context;
always be a relative
of course there may
absolute one. Because
is, we suggest, an
(for example children)
difficult for learners
in that what is
difference of awareness
adults).
advanced (for example
,i'cor those who are
may be simpli!
attempts to bring
records continuing
readability research
of
The history of
to be a function
in other words is seen
the reader. Readability
for
closer the text and
of comprehension
difficulty for texts and
that measures of
(1957)
their interaction so
of the doze technique
Taylor's development
which
readers are both necessary.
in onc measure
these two variables
attempt to combine
focusing on
was a deliberate
Davies (1984),
of newspapers.
ascertain readability
original text
could be used to
experiment in which an
situation, reports on an
comprehension by a
the second language
in terms of their
version were comparcd
and its simplified
linguistically
that the
English. Thc hypothesis was
Japanese teachers of
hypothesis
group of
text. The
better than the original
be comprehended
simpler text would
102
7
"V"Irryttfrot,
verbatim
In that experiment the measure of comprehension was
was supported.
doze.
obviously be taken
In our current task of writing a dictionary, care must
explanations can be understood by the reader. That means
that the definition or
carefully so that the explanations have appropriate
targeting the dictionary
have a built-in pedagogic
readability for their audience. Dictionaries therefore
what a dictionary is and in particular what
function. This raises the questions of just
by
needed for a professional-academic audience. Some views
sort of word-book is
be of interest:
dictionary makers will
that 'A dictionary of
Abercrombie N., Hill S., Turner B S (1984) claim
definitions, but inevitably a statement of what
sociology is not just a collection of
development and
It is also prescriptive in suggesting lines of
is.
the discipline
subject as diverse and dynamic as
consolidation. The problem of definition in a
between an existing consensus, however fragile and
sociology is to strike a balance
potential. The unifying theme of this dictionary is our
temporary, and a developing
elaborated and vital discipline within
conviction that sociology is an autonomous,
than
Our enthusiasm for the subject was sustained rather
the social science corpus.
precision within the conflicting range of
diminished by the experience of seeking
sociology.' (p. vii). 'A statement of what the
perspectives that constitute modern
but nevertheless inevitably what all dictionary
discipline is': a tall order indeed
role.
making assumes in its normative
Dictionat y is written
West and Endicott (1935159) maintain 'This English
words 'which be knows the
specially for the foreigner. It explains to him in
does not know.' (p. iii). In words which
meaning of words and idioms which he
welcomed linguistic straitjacket of
s/he knows the meaning of, stresses the
dictionary making.
intended as an at-band
Angeles (1981) states that his dictionary 'is
used as a supplement to
reference for students, laypersons, and teachers. It can be
be consulted for philosophy's own
texts and philosophy readings; it can also
Angeles refers to
enjoyment and enlightenment.' (p. ix). Even the 'laypersons'
Audience is critical and
must surely be informed, interested, educated and so on.
simplification, if
when it includes students necessarily demands some measure of
4
not of language, certainly of substance.
103
.)
in the
of Language Testing by colleagues
The idu for writing a Dictionary
out of several
Centre, University of Melbourne arose
NLL1A Language Testing
all know what we are
in-house set of glosses so that we
needs; the need for a kind of
compromise
register; then, as with text-books, the necessary
talking about, our own
Applied Linguistics and
like ourselves working in
between the profession (those
those with general
Such a compromise targets
languago testing) and the public.
disciplines. Very
ie MA students of the relevant
,.ther than specialist knowledge,
Weber in their
targeted by Richards, Platt and
much, in fact, like the audience
that working
Linguistics (1985). We have found
Longman Dictionary of Applied
educating one another,
felicitous way of sharing and
together on this dictionary is a
need for our
and creates the very register we
precisely because it defines, explores
McNamara, Cathie
dictionary (Alan Davies, Tim
work. Those working on the
Hock), are all very
Lumley, Chris Corbel, Yap Soon
Elder, Annie Brown, Tom
The Longman
that this is a long term task.
much part-time and we.are aware
making. A larger
after all, was 4 years in the
Dictionary of Applied Linguistics,
marshalling and
provides wide coveragt but needs
contributing group of authors
organisation is essential.
organising. And in such an exercise
audience and
the problems have to do with
As I have already suggested
format of
selection and coverage, scope and
definition; but equally important arc
have been helped by
reach agreement over these matters we
entry. In attempting to
lexicography. If indeed what
not at all new in
the realisation that such concerns are
lexicography.
we arc doing is
Kilpfer's Workbook (1984:1) 'A dictionary is a
Let me quote from
variety, usually
of a language or language
reference book containing the words
functions,
their forms, pronunciations,
alphabetically arranged, with information on
reference book
dictionary may be more than a
meanings and idiomatic uses. A
knowledge as well as
biographical and geographical
about words; it can contain
symbols; and the
weights and measures, and
lists of colleges and universities,
dictionary making.
about aspects of language and
introduction may include articles
and meanings, but
contain not only pronunciations
The entries themselves may
information an
and even the kind of
information about grammar and usage
word names.'
encyclopedia gives about the thing the
writing a
type of ambition: are we
There are indeed many terms for our
Opitz writes of
list, a glossary, a reference list?
dictionary, an encyclopedia, a word
of
glossary, perhaps -ie a list
dictionary', but is that what it is or a
a 'segmental
which is what
`to isolate a distinct register'
technical terms rather than an attempt
(Opitz 1983: 58).
Opirt means by a segmental dictionary.
104
But is it a glossary? Hartmann defines glossary as a 'word-list with
explanation of meanings' (Hartmann 1983: 223). Moulin describes a glossary as a
details two techniques of
go of glosses appended to text, often specialised, and
ordering, by areas of interest and by alphebet: 'most authors (of specialist
dictionaries) are neither linguists nor professional lexicographers, but specialists in
the particular discipline...these glossaries are commissioned...to try and introduce a
facilitate the
ineasure of normalisation in the use of specialist terms and thus
exchange of information' (Moulin 1983: 146). We will return to that concern for a
'measure of normalisation'.
Is it an encyclopedia? Hartmann tells us that encyclopedic information has
lexical information' (Hartmann
to do with 'practical knowledge of things versus
1983: 223). A more elaborate distinction is made by Read (1976: 713f0 quoted in
McArthur 1986: 'The distinction between a dictionary and an encyclopedia is easy
words,
ro state but difficult to carry out in a practical way: a dictionary explains
whereas an encyclopedia explains things. Because words achieve their usefulness
by referring to things, however, it is difficult to construct a dictionary without
considerable attention to the objects and abstractions designated'.
McArthur reminds us that the Encyclopedia Britannic& bad its origin in
Edinburgh. Notice its original title: 'The Encyclopedia Britannica or a Dictionary
of Arts and Sciences, compiled upon a New Plan' (Edinburgh 1768-71, sponsored
by the Society of Gentlemen in Scotland). The Britannica was a very obvious
product (no doubt influenced by the French philosophers) of the Scottish
Enlightenment, that high point in Scottish history, when Scotland truly was the
clever country. From that high point we are brought down to earth by the comment
of William Smellie, one of the original authors: 'with pastepot and scissors I
composed it' (W. Smellie in Kogan 1956: 14, quoted in McArthur: 106-7).
McArthur suggests as a way of resolving the overlap in the uses of the
terms Dictionary and Encyclopedia that it is probably best not to bother. 'The
simplest way' he says 'of resolving the tension seems to be to accept the way in
which the early encyclopedists handled the matter. In this dilemma we in fact work
along a continuum rather than within separate containers, where one extreme is
words and words alone, and the other is referents and referents alone' (McArthur
1986: 104). At one end of McArthur's continuum is the dictionary, at the other the
encyclopedia and in between the encyclopedic dictionary.
McArthur suggests as a way of relating dictionaries and encyclopedias
(which in the USA and France, but not in the UK, have, he says always been linked)
the following pair of ternr `thai could be useful in studying the world of reference
materials:
105
of words and the
with the world
which deals
micro-lexicography,
alphabetic dictionary).
1.
instances is an
(which in most
of things and
into the world
wordbook proper
which shades out
instances are
(which in most
macro-lexicography,
2.
of knowledge
on compendia
alphabetic') (McArthur
subjects, and centres
nowadays are also
in most instances
encyclopedias, which
1986: 109).
two well-
Applied Linguistics
the field of
attempts within
First
Linguistics and the
When we look at
Dictionary of Applied
the Longman
their Longman
known products are
Introduction to
Phonetics. In the
Linguistics and
(1985) ask who is this
Dictionary of
Platt and Weber
Linguistics, Richards,
of Applied
for students
Dictionary of Applied
that it is intended
and in the
for and conclude
both in training
dictionary intended
language teachers
General Linguistics;
Linguistics and
field. (1985: v).
which
simple definitions
produce clear and
'has been to
language.
'Our aim' they say
in non-technical
meanings of a term
links to
basic and essential
references show
communicate the
possible, but cross
self-contained as far as
fuller
information where a
Definitions are
references provide
concepts, and
other terms and
found.' (p. vii).
concept can be
discussion of a term or
in the
is Crystal who
with the problems
in our grappling
(1980) confesses 'I
More helpful to us
and Phonetics
of Linguistics
for this book is
First Dictionary
Preface to his A
appropriate title
the most
less
now whether
by themselves were
remain doubtful even
of the entries
definitional parts
necessary to
"dictionary". The
consequently it proved
expected; and
illustrations, to
might have
illuminating than one
with several
discursive approach,
a more
encyclopedic
introduce in addition
accordingly contain
Most entries
significance of' a term.
term was used,
context in which a
capture the
the historical
such matters as
fields' (p. 5).
information about
from associated
term and others
rOationship between a
or the
that is there arc no
'is self-contained:
Crystal continues,
exposition of a sense.
'Each entry',
complete the
other entries to
term...I have
cross-references to
looking up a
obligatory
'See Y' after
of the convention
dictionary
dictionary-users open a
Nor have I made use
that, as most
the principle
satisfactory account
preferred to work on
be given a
mind, they should
under
problematic term in
explain competence.
with a single
possible. I therefore
on. As a
immediately as
PERFORMANCE, and so
of that term as
performance under
procedure means
COMPETENCE,
however, this
of these terms,
term
characteristics of the
the interdependence
consequence of
the salient
repetition: at least
vice-
be some
COMPETENCE, and
that there must
into the entry for
be incorporated
performance must
106
weakness, if the book were read from cover to
versa. This repetition would be a
text-book, and while the result bas
cover; but a dictionary should not be used as a
been the case if the 'Sec..'
been a somewhat longer volume than would have
convention had been used, I remain convinced of the greater benefits of ,Jok-up
convenience and entry coherence.' (p. 5).
After some preliminary trials, pilot entry writing and a small-scale survey
determined on the following
of the entries among teachers and MA students, we
guidelines for ourselves:
the entries should be on the encyclopedia side of McArthur's continuum,
1.
explaining where appropriate;
more than language-definitional,
they should where possible (and appropriate) give examples so as to situate
2.
rbe explanation;
they should accept overlap, in Crystal's sense, so that referring to other
3.
where necessary for
entries for necessary explanation would be avoided, except
informative purposes; citations would be minimised except in the sense of the
informative.purpose above;
in other words coming
where possible one clear definition should be attempted,
descriptive. We have taken the
down on the side of being normative rather than
OED) it is our role to
view that unlike a truly descriptive dictionary (such as the
contain and confine to 'try and introduce a measure of normalisation in the use of
specialist terms and thus facilitate the exchange of information.' (Moulin 1983:
146).
Whether what we are doing therefore should be called a dictionary or an
encyclopedia is really beside the point. But while it does veer towards the
encyclopedia side of the McArthur continuum it retains important aspects of
dictionary-ness. It does attempt definitions, it avoids essays (so it is not a Glossary
'I have retained the procedure of organizing the Glossary as a series of
either:
essays' (Abrams 1981: v) but unlike many dictionaries it has no information of a
pronunciation kind (though obviously it would not eschew this where it seemed
relevant) nor does it systematically contain historical material about derivations. So
it probably is what McArthur calls an encyclopedic dictionary.
We are in our Language Testing Dictionary concerned to establish a
uniform style of entry and at the same time to ensure adequate coverage. To
illustrate these questions and through them the importance in our view of being
more encyclopedic than dictionary-like, I turn now to a comparison of alternative
entries.
9
107
Given
in our dictionary.
hundred entries
for about four
present view is very
We are planning
version below, our
version and the B
would permit a
between the A
A version
the choice
though use of the
B
of the B version, even
shorter than the
much in favour
version is much
each case the A
the A
of entries. In
encyclopedic like and
larger number
version is more
therefore the B
below our
version, in some sense
comparisons reported
making the
dictionary like. In
the B
of this Dictionary
version more
perceived nature
because of our
hypothesis was that
than the A versions.
likely to be readable
versions were more
Results
Experiment and
entries
read three sets of
21) were asked to
students (N
short or
A class of MA
they too long, too
(1) their length - were
and comment on
about right.
difficult, too easy or
(see Appendix)
were they too
(2) their difficulty -
to have to make.
about right; and
unsatisfactory choices
these were
with
is apparent that
do provide us
With hindsight it
Nevertheless the responses
'too easy' mean?
provided.
What after all does
versions we had
of the conuasting
of the readability
some indication
the basis
and (b) versions on
between the (a)
was made
indication of the ratio
Next a comparison
density is an
1985). Lexical
density (Halliday
that in informal
of their lexical
Halliday reports
clause by clause.
granunatical loading
it is typically
of lexical to
written language
about 2; in adult
lexical density is
high as 10-13
spoken English
it can be as
In scientific writing
about 6 per clause.
difficult except to
more dense, say
writing is often so
why scientific
almost
That is one reason
headlines can be
per clause.
newspaper
explanation of why
is also an
the expert. It
issue.
is currently at
know exactly what
uninterpretable, unless you
the
students alongside
of the Masters'
summed responses
Here are the
finding for each entry.
lexical density
108