Modeling with Words. Learning, Fusion and Reasoning within a Formal Linguistic Representation Framework

240 Pages·2.964 MB·English

by  Lawry J., Shanahan J., Ralescu A. (eds.)

Foreword
Scientific progress is driven, in large measure, by questioning the validity of
axioms,dogmasandtraditions.Oneofthemostdeep-seatedtraditionsinscience
is that of according much more respect to numbers than to words. The essence
of this tradition was articulated succinctly by Lord Kelvin in 1883:
In physical science the first essential step in the direction of learning
any subject is to find principles of numerical reckoning and practicable
methods for measuring some quality connected with it. I often say that
when you can measure what you are speaking about and express it in
numbers, you know something about it; but when you cannot measure
it,whenyoucannotexpressitinnumbers,yourknowledgeisofameager
and unsatisfactory kind: it may be the beginning of knowledge but you
havescarcelyinyourthoughts,advancedtothestateofscience,whatever
the matter may be.
Adherencetothetraditionofprimacyofnumbersoverwordshasledtobrilliant
successes that are visible to all. But does it follow that the tradition should be
accorded this status of immutable truth? No, certainly not.
The reality is that alongside brilliant successes we see sobering failure. We
have sent men to the Moon but we cannot build a robot that can play tennis.
We canbuild computers thatcanexecute billions ofinstructions per secondbut
cannotcreateprogramsthatcansummarizeanon-stereotypicalstory,muchless
abook.We cannotautomatedrivingincitytrafficandwecannotconstructma-
chinetranslationprogramsthatcanperformatthelevelofahumaninterpreter,
and so on. Does this reality suggest that the tradition of primacy of numbers
over words may be a part of the problem? In a sense, ‘Modelling with Words’
may be viewed as an affirmative answer to this question.
Inretrospect,my1973paper,‘Outline ofaNewApproachtotheAnalysisof
Complex Systems and Decision Processes,’may be seen as an initial step in the
direction of enhancing the power of words through the use of words as objects
of computation. A basic concept that was introduced in that paper is that of a
linguistic variable, that is, a variable whose values are words or phrases drawn
from a naturallanguage.The concept of a linguistic variable is closely linked to
the concept of granulation and forms the basis for the concept of a linguistic if
– then rule.
Reflecting the depth of the tradition of the primacy of numbers over words,
the initial reaction to the concept of a linguistic variable ranged from mixed to
skeptical and hostile. Commenting on my first presentation of the concept of a
linguistic variable in 1972, Professor Rudolf Kalman, a brilliant mind, had this
to say:
I would like to comment briefly on Professor Zadeh’s presentation. His
proposals could be severely, ferociously, even brutally criticized from a
VI Foreword
technical point of view. This would be out of place here. But a blunt
questionremains:IsProfessorZadehpresentingimportantideasoris he
indulging in wishful thinking? No doubt Professor Zadeh’s enthusiasm
for fuzziness has been reinforced by the prevailing climate in the U.S. –
oneofunprecedentedpermissiveness.‘Fuzzification’isakindofscientific
permissiveness;it tends to resultin socially appealing slogansunaccom-
panied by the discipline of hard scientific work and patient observation.
Today, almost all applications of fuzzy logic and fuzzy set theory involve the
concept of a linguistic variable. Underlying these applications are two principal
rationales.Thefirst,andmostobvious,isrelatedtoproblemsinwhichthevalues
of variables and their interrelations are not known with sufficient precision to
justifytheuseofnumbers.However,itshouldbenotedthatitiscommonpractice
to employ numbers, even when their use is not justified, for the simple reason
thatnumberscarrymorerespectthanwords.Thesecondrationalerelatestothe
useofwords–sincewordsareintrinsicallylessprecisethannumbers–toexploit
the tolerance for imprecision to achieve tractability, robustness and, above all,
low solution cost. It is this rationale that underlies the extensive use of fuzzy
logic in consumer products and industrial systems.
Seenagainstthis backdrop,‘Modelling with Words’maybe viewedas a pre-
sentation of a persuasive ease for the development and adoption of the method-
ology of linguistic modelling – a methodology that is an extension and gen-
eralization of conventional, numerically-based approaches to modelling. In this
perspective, the machinery of modelling with words is an important addition to
the armamentarium of modelling methodologies.
‘Modelling with Words’is animpressive collectionofauthoritative contribu-
tions that relate to a wide range of issues in modelling and systems analysis,
extending from conceptual graphs and fuzzy quantifiers to humanist computing
and self-organizing maps. There is much that is new in Modelling with Words,
and numerous examples add significantly to the value of the information.
A basic issue, which is addressed by some of the contributors, is the rela-
tionship between fuzziness and randomness. This issue has been an object of
discussion and debate since the early days of fuzzy set theory and fuzzy logic.
The crux of the issue is the question:‘Is fuzziness distinct fromrandomness?’A
historyofdiscussionsofthisissuemaybefoundinmy1995paperinTechnomet-
rics entitled ‘Probability Theory and Fuzzy Logic Are Complementary Rather
than Competitive.’
It was pointed out by Robbins in 1965, Loginov in 1966, Orlov in 1980,
and many others since then that a fuzzy set may be generated by a random
set. A physical analogy that is described in my 1995 paper is the following. A
fuzzy shadow on the base of an enlarger may be generated by dodging, that
is, by a pattern of randomly moving an opaque planar object under the lens
of the enlarger. However, the same shadow can also be generated in a non-
random way by placing under the lens a stack of partially translucent sheets
shaped like the α-cuts of the fuzzy shadow. Thus, what can be said is that a
fuzzy set may be generated from a family of crisp sets by various operations,
Foreword VII
some randomand some not.Furthermore,anything thatcanbe done with crisp
random sets can be done with convex combinations of crisp non-random sets.
The mathematical linkage between fuzzy sets and random sets may be used to
obtain useful results, as was done in the books by Orlov and Goodman, and
by the authors in ‘Modelling with Words.’ However, the mathematical linkage
between fuzzy sets and random sets does not suggest that fuzzy set theory is
a part of probability theory or vice-versa. It may be argued, as I have done in
my 1995 paper, that the two theories are complementary, but my more recent
andmore radicalview is thatprobability theoryshouldbe basedonfuzzy logic,
rather than on the bivalent logic that it has been based on since its inception.
AnotherissuethatIshouldliketocommentonisthe relationbetweenMod-
elling with Words (MW) and Computing with Words (CW).
Asitslabelsuggests,inComputingwithWordstheaccentisoncomputation
rather than on modelling. Thus, in CW the objects of computation are words,
phrases or propositions drawn from a natural language. A major focus of at-
tention in CW is computation with perceptions, with the understanding that
perceptionsaredescribedbypropositionsdrawnfromanaturallanguage.Akey
component of CW is Precisiated Natural Language (PNL).
TherelationshipbetweenModellingwithWordsandComputingwithWords
is close and is likely to become evencloser with the passageof time. In the final
analysis, both are aimed at enlarging the role of natural languages in scientific
theories and, especially, in knowledge management, decision and control.
‘Modelling with Words’ is an important contribution to the conception, de-
sign and utilization of intelligent systems. It is informative, authoritative and
reader-friendly.The editors,J.Lawry,J.ShanahanandA. Ralescu,the contrib-
utorsandthepublisher,Springer-Verlag,havedoneanexcellentjobanddeserve
our thanks and congratulations.
July 2003 Lotfi A. Zadeh
Berkeley, CA
VIII Foreword
Preface
The development of high-performance computers and the corresponding ad-
vances in global communications have lead to an explosion in data collection,
transmissionandstorage.Large-scalemultidimensionaldatabasesarebeinggen-
erated to describe a wide variety of systems. These can range from engineering
applications such as computer vision, to scientific data such as that from the
genome project, to customer and price modelling in business and finance. In
all of these cases the data is useless without methods of analysis by which we
can discover the important underlying trends and relationships, integrate other
backgroundinformation,and then carryout inference on the learntmodels. For
a number of reasons we argue that in order to fulfill these requirements we
should move towards a modelling paradigm that is as close to natural language
as possible.
Inrecentyearsthe areaof machine learning has focusedon the development
of induction algorithms that is maximize predictive accuracy. However, since
there has been little emphasis on knowledge representation the models derived
are typically ‘black box’and therefore difficult to understandand interpret. For
many applications a high level of predictive accuracy is all that is required.
However,in a largenumber ofcases,including many criticalapplication,a clear
understanding of the prediction mechanisms is vital if there is to be sufficient
confidenceinthemodelforittobeusedasadecision-makingtool.Modeltrans-
parency of this kind is best achievedwithin a natural-language-basedmodelling
framework that allows for the representation of both uncertainty and fuzziness.
Wemustbeaware,however,ofaninherenttrade-offbetweenmodelaccuracyand
transparency. Simple models, while the most transparent, are often inadequate
to capture the complex dependencies that exist in many practical modelling
problems.Alternatively, more complex models are much more difficult to repre-
sent in a clear and understandable manner. This trade-off is best managed by
close collaboration with domain experts who can provide the modeller with an
unbiased assessment of the transparency of their models while also establishing
what level of accuracy is necessary for the current problem. Another important
justification for learning models at a linguistic level is that it facilitates their
fusion with backgroundknowledge obtained from domain experts.
In any data modelling problem there is almost certain to be some expert
knowledge available, derived from either an in-depth understanding of the un-
derlying physical processes or from years of practical experience. In expert sys-
tems the emphasis is placed almost entirely on this expert information, with
data being used only to optimize the performance of the model. On the other
hand,inmachinelearning,backgroundknowledgeislargelyignored,exceptper-
haps in the limited roleofconstrainingpriordistributions inBayesianmethods.
As part of modelling with words we propose that there should be a high-level
fusion of expert- and data-derived knowledge. By integrating these two types
X Preface
of information it should be possible to improve on the performance of models
thatarebasedsolelyononeorthe other.Furthermore,theeffectiveuseofback-
ground knowledge can allow for the application of simpler learning algorithms,
producing simpler, and hence more transparent, models.
Given a model of a data problem it is highly desirable that practitioners be
able to interrogate it in order to evaluate interesting hypotheses. Since these
hypotheses are most likely to be in natural-language form, to achieve this a
high-levelinferencemechanismonlinguistictermsisrequired.Suchaninference
process is, in essence, what Zadeh calls ‘computing with words.’ The nature of
any reasoning mechanism at this level will depend on the nature of the data
models.Forexample,ifthemodelstaketheformofafuzzyrulebasethenmeth-
ods similar to those proposed by Zadeh may be appropriate. Alternatively, if
the model consists of conceptual graphs then graph matching and other similar
methodsfromconceptualgraphtheorywillneedtobeused.However,nomatter
what methodology is applied it must be formally well defined and based on a
clearunderlyingsemantics.Inthisrespectmodellingwithwordsdiffersfromnat-
urallanguagesincewerequireamuchmoreformalrepresentationandreasoning
frameworkfortheformerthanforthelatter.Infactthishighlevelofformalrigor
is necessary if we are to obtain models that are sufficiently transparent to sat-
isfy practitioners of their validity in critical applications. Certainly, a modelling
processcannotbe truly transparentif there aresignificantdoubts regardingthe
meaning of the underlying concepts used or the soundness of the learning and
reasoning mechanisms employed. This formal aspect of modelling with words
is likely to mean that some of the flexibility and expressiveness of natural lan-
guage will need to be sacrificed. The goal, however, is to maintain rigor within
a representationframeworkthatcaptures manyofthe importantcharacteristics
of natural language so as to allow relative ease of translation between the two
domains. This is very similar to the idea behind Zadeh’s ‘precisiated natural
language.’
Modelling with words can be defined in terms of the trilogy,learning, fusion
andreasoningascarriedoutwithinaformallinguisticrepresentationframework.
As such this new paradigm gives rise to a number of interesting and distinct
challenges within each of these three areas. In learning, how can the dual goals
ofgoodpredictiveaccuracyandahighleveloftransparencybereconciled?Also,
how can we scale our linguistic algorithms to high-dimensional data problems?
In fusion, what are the most effective methods for integrating linguistic expert
knowledgewithdata-derivedknowledge,andhowdoesthisprocessconstrainthe
representationofboth types ofknowledge?In reasoning,what soundand useful
rules of inference can be identified and what type of queries can they evaluate?
In general, how can we effectively integrate fuzzy and probabilistic uncertainty
indatamodellingandwhattypeofknowledgerepresentationframeworkismost
appropriate? This volume contains a collection of papers that begin to address
someoftheseissuesindepth.PapersbyE.Hernandezetal.andA.Laurentetal.
investigatethe useoffuzzy decisiontreesto derivelinguistic rulesfromdata.H.
Ishibuchietal.andR.Alcalaetal.describehowgeneticalgorithmscanbe used
Preface XI
toimprovetheperformanceoffuzzymodels.Theareaoffuzzyconceptualgraphs
is the topic of papers by T. Cao and P. Paulson et al. Linguistic modelling and
reasoningframeworksbasedonrandomsetsarediscussedinpapersbyJ.Lawry
and F. Diaz-Hermida et al., and Q. Shen introduces an algorithm according to
which rough sets can be used to identify important attributes. The application
of fuzzy sets to text classification is investigated by Y. Chen, and J. Rossiter
discussesthe paradigmofhumanistcomputing andits relationshiptomodelling
with words.
June 2003 Jonathan Lawry
Jimi Shanahan
Anca Ralescu
XII Preface
Author Index
Alcala´, Rafael ...................44 Ishibuchi, Hisao ...............209
Barro, Sen´en .....................1 Laurent, Anne .................102
Bouchon-Meunier, Bernadette .102 Lawry, Jonathan ...............186
Bugar´ın,Alberto .................1
Marsala,Christophe ...........102
Cao, Tru H. .....................80
Paulson, Patrick ...............168
Carin˜ena, Purificacio´n ............1
Recasens, Jordi .................26
Chen, Yi-Ping Phoebe .........153
Rossiter, Jonathan .............124
Cordo´n, Oscar ..................44
Shen, Qiang ....................64
D´ıaz-Hermida, Felix ..............1
Tzanavari,Aimilia .............168
Herna´ndez, Enric ...............26
Herrera, Francisco ..............44 Yamamoto, Takashi ............209