Exemplar-Based Knowledge Acquisition. A Unified Approach to Concept Representation, Classification, and Learning

173 Pages·1989·10.24 MB·English

Exemplar-Based
Knowledge Acquisition
A Unified Approach to
Concept Representation,
Classification, and Learning
Ray Bareiss
Department of Computer Science
Vanderbilt University
Nashville, Tennessee
ACADEMIC PRESS, INC.
Harcourt Brace Jouanovich, Publishers
Boston San Diego New York
Berkeley London Sydney
Tokyo Toronto
For Audrey
Copyright © 1989 by Academic Press, Inc.
All rights reserved.
No part of this publication may be reproduced or
transmitted in any form or by any means, electronic
or mechanical, including photocopy, recording, or
any information storage and retrieval system, without
permission in writing from the publisher.
ACADEMIC PRESS, INC.
1250 Sixth Avenue, San Diego, CA 92101
United Kingdom Edition published by
ACADEMIC PRESS INC. (LONDON) LTD.
24-28 Oval Road, London NW1 7DX
Library of Congress Cataloging-in-Publication Data
Bareiss, Ray.
Exemplar-based knowledge acquisition : a unified approach to
concept representation, classification, and learning / Ray Bareiss.
p. cm. - (Perspectives in artificial intelligence : v. 2)
Bibliography: p.
Includes index.
ISBN 0-12-078260-X (alk. paper)
1. Categorization (Psychology) - Data processing. 2. Mental
representation - Data processing. 3. Learning, Psychology of - Data
processing. 4. Protos (Computer program) 5. Machine learning.
6. Artificial intelligence. I. Title. II. Series: Perspectives in
artificial intelligence : vol. 2.
BF445.B37 1989
153.2'3'02855369 - dc20 89-15198
CIP
Printed in the United States of America
89 90 91 92 9 8 7 6 5 4 3 2 1
For Audrey
Copyright © 1989 by Academic Press, Inc.
All rights reserved.
No part of this publication may be reproduced or
transmitted in any form or by any means, electronic
or mechanical, including photocopy, recording, or
any information storage and retrieval system, without
permission in writing from the publisher.
ACADEMIC PRESS, INC.
1250 Sixth Avenue, San Diego, CA 92101
United Kingdom Edition published by
ACADEMIC PRESS INC. (LONDON) LTD.
24-28 Oval Road, London NW1 7DX
Library of Congress Cataloging-in-Publication Data
Bareiss, Ray.
Exemplar-based knowledge acquisition : a unified approach to
concept representation, classification, and learning / Ray Bareiss.
p. cm. - (Perspectives in artificial intelligence : v. 2)
Bibliography: p.
Includes index.
ISBN 0-12-078260-X (alk. paper)
1. Categorization (Psychology) - Data processing. 2. Mental
representation - Data processing. 3. Learning, Psychology of - Data
processing. 4. Protos (Computer program) 5. Machine learning.
6. Artificial intelligence. I. Title. II. Series: Perspectives in
artificial intelligence : vol. 2.
BF445.B37 1989
153.2'3'02855369 - dc20 89-15198
CIP
Printed in the United States of America
89 90 91 92 9 8 7 6 5 4 3 2 1
Editor's Note
As most of us in the field are well aware, Artificial Intelligence has no universally accepted
research methodology. One time-honored method of progress is to define a problem in
formal terms and propose and analyze classes of solutions formally, with logic being the
formalism of choice in AI. When this is done right, there is a clear sense of a "result" that
can be shared and built upon. There are several ways, however, in which solutions of this
type may not constitute much of an advance:
The problem is oversimplified at the definition stage, in an attempt to simplify it suffi­
ciently for analysis, and the really interesting aspects of the problem are "assumed
away", for example, how real-world cognitive agents actually solve the problem.
The class of solutions is so abstract that at the end of the exercise we still do not know
how to implement the solutions computationally, and the formalism is not particularly
helpful in making the additional choices necessary to actually implement the solution.
The class of solutions is so intractable computationally that the true source of power
in solving problems of this type in the real world remains hidden.
Another method of making progress is to display a computer program that solves some
version of the problem. The problem here is, of course, that it is often difficult to know
what the "result" is that succeeding generations can use. Sometimes, when a researcher
builds such a system and then claims that the success is due to property A of the imple­
mentation approach, it often turns out on later analysis that property A was merely an
incidental and largely irrelevant player and that some other aspect of the implementa­
tion was, in fact, more central to the performance of the system. Because of these uncer­
tainties about what the exercise means, why certain commitments were made, and what
alternative commitments could have been made, experiments of this type are often
described as "ad hoc" by critics. While some of this research is indeed ad hoc, quite a
large body of this type of research is actually based on some principled theory of the
underlying phenomenon, but since this theory is not "formal", people often find it diffi­
cult to understand the motivations behind the commitments of the theory.
These different styles within AI research have been characterized as the "neats" versus
the "scruffies" If real cognitive systems are computational systems with many "moving
parts", i.e., to describe and understand the sources of their power adequately a nontrivial
number of commitments have to be made simultaneously, then theory-making in AI for
complex cognitive phenomena is bound to traffic in complex models, i.e., in models that
look scruffier than we might like. However, the opportunity to make theories with many
moving parts is also a danger: Occam's razor is often thrown out of the window, and
researchers come up with different complex systems, and often there is no methodolgy
for comparing them. The researcher may simply keep adding elements in a manner that
appears ad hoc to outsiders who do not share his intuitions. This results in subgroups
who share each other's intuitions and motivations and thus in fact critique each other's
proposals, but increasingly they can only talk to the other members of the subgroup. This
is the origin of much of the fragmentation of research methodologies in AI. Thus the
scruffies, of which I count myself as one, have a methodological problem.
What impresses me about Ray Bareiss' dissertation, a modified version of which you
hold in your hands, is that it tackles a problem of significant intrinsic complexity. Formal
theories are clearly inadequate to capture the variety in real-world versions of the phe­
nomenon. Bareiss' approach seems to me to be a paradigm of how to do good scruffy
science: He combines computational experimentation and real-world psychological
data and insights derived therefrom with the precision of definitions and clear argumen­
tation. This helps him to minimize the arbitrariness of assumptions that often dog the
scruffy AI scientist. Bareiss' choice of the problem of classification is an excellent one for
this purpose: Extensive psychological literature is available with real-world data, the
problem has a number of formal elements in it that help in precise statements of the
problem and solution features, and, finally, the solutions can be tested in the "real-world"
of AI as knowledge acquisition tools for knowledge-based systems.
One of my goals for the Perspectives in AI Series is for it to be a showcase for research
that aims at the middle ground between formalistic approaches that do not handle the
complexity of real-world cognition and engineering approaches that do not help us
understand intelligence as a phenomenon. Ray Bareiss' work seems to me to be an
excellent example of this type of work.
- B. Chandrasekaran
Acknowledgments
I would like to thank Bruce Porter and Craig Wier for several years of
rewarding collaboration on the Protos Project. My many friends and col­
leagues at the University of Texas made this research possible. Unfortu­
nately, there are too many to detail their assistance over the years, so I will
merely list their names: Agnar Aamodt, Brad Blumenthal, Karl Brant-
ing, Bob Causey, Larry Clay, Dan Dvorak, Adam Farquhar, Rob Holte,
Ben Kuipers, Vipin Kumar, James Lester, Rich Mallory, Ray Mooney, Ken
Murray, Gordon Novak, Claudia Porter, Joe Ross, Bob Simmons, and Art
Souther. I would also like to thank the reviewers for their comments. This
book was typeset with I^TjrX by Yvonne van Olphen and the author.
Support for this research was provided by the National Science Founda­
tion under grant number IST-8510999 and the Army Research Office under
grant number ARO DAAG29-84-K-0060.
ix
Preface
How is memory organized so that past experiences can be used to improve
problem solving? An important organizing principle is that common ex­
periences are collected into categories. Despite its simple description, this
principle is surprisingly difficult to implement computationally.
The basis for category membership is often unclear, a problem that has
long been recognized. For example, what unites the members of the cat­
egory "science?" Some members—such as physics and biology—are clear
exemplars. Others—such as computer science and cognitive science—are
less clear; arguments can be made that either support or reject their mem­
bership. Forming a category, and classifying new cases as belonging to
that category, depends on determining commonalities and reasoning with
arguments.
This problem is not merely a source of academic curiosity. It surfaces
in the "real world." For example, in medicine, clinicians must diagnose
patients' disorders without strict definitions of each disorder. Descriptions
of patients' disorders within each category vary widely, and accurately as­
signing new cases to the categories requires solving the problem of forming
and using ill-defined categories.
Ill-defined categories arise when a domain has a weak theory. For these
domains, the criteria for category membership are incomplete or too costly
to apply. Categories have a graded structure of central, paradigmatic ex­
emplars and an assortment of less clear, but arguable, exemplars. This
manuscript describes a program, Protos, that learns such categories and
classifies new cases by explaining their similarities to known exemplars.
Protos began as a computational model of psychological theories of con­
cept learning and classification. Drawing on research by Smith, Medin,
Murphy, and others, we designed Protos to demonstrate exemplar-based
learning and classification. In addition, Protos is a practical tool for con­
structing knowledge-based programs.
Protos was applied to the task in clinical audiology of diagnosing pa­
tients' hearing disorders from symptoms and test results. Through direct
xi
Xll Preface
interaction with a domain expert, it achieved expert-level proficiency. Pro­
tos is an effective knowledge-acquisition tool, supporting both construction
and refinement of knowledge bases. One version of Protos has been widely
distributed and is being applied to numerous domains.
This research contributes to our understanding of fundamental issues
in cognitive science, and it develops technology for solving real problems.
In summary, it is a tribute to the artificial-intelligence methodology of
experimentation, rather than speculation.
The University of Texas Bruce W. Porter
May, 1989
Chapter 1
Introduction
When these fellows are at fault, they come to me, and I generally
manage to put them on the right scent. They lay all of the
evidence before me, and I am generally able, by the help of my
knowledge of the history of crime, to set them straight. There is
a strong family resemblance about misdeeds, and if you have the
details of a thousand at your finger ends, it is odd if you can't
unravel the thousand and first.
— Sherlock Holmes [Doy60]
1.1 Preliminaries
Classification is fundamental to intelligent problem solving. By classifying
a new entity {i.e., object, event, or state), a problem solver can avoid much
of the effort required for processing a unique occurrence. The problem
solver's past experience with similar entities can be brought to bear, and
its limited computational resources conserved.
Classification is the process of recognizing an entity as an instance of
a known concept. A concept is the intension al representation of a class of
entities which are equivalent with respect to the problem solver's goals and
experience. An entity is classified by recognizing that its attributes imply
that it is an instance of a particular concept more strongly than they imply
that it is an instance of any contrasting concept. To classify reliably, the
problem solver must have an accurate representation of the correspondences
between attributes and concepts.
Reliable acquisition of concepts is the primary prerequisite for classifica­
tion. Traditionally, knowledge-based systems have been given hand-crafted
1
2 1. INTRODUCTION
conceptual structures for their intended domains. Although this approach
is adequate for some tasks, such a system tends to be brittle because its
static knowledge cannot accommodate changes in the domain [H0I86]. An
alternate approach is to integrate learning with problem solving, thus al­
lowing the system to learn new concepts and to refine existing ones in light
of its experience.
1.2 The Goal of this Research
The goal of this research was to produce and validate a system that learns
to perform accurately and efficiently in the normal course of its use for
classification. A human teacher presents the system with, cases to be clas­
sified and provides explanations of cases that the system fails to classify
correctly. Through this training, the system acquires concepts, builds an
indexing structure, and acquires a general body of domain knowledge. Con­
sequently, the system learns to classify accurately and efficiently and is able
to explain its classifications.
1.3 Constraints on Concept Learning
Concept learning has been the primary focus of machine learning research.
For example, Machine Learning: An Artificial Intelligence Approach, Vol
ume 2 [MCM86] cites 86 publications on this topic. Yet in spite of the
magnitude of this research effort, the problem is not regarded as solved.
The environment in which a concept learner functions imposes con­
straints on its design which have not been thoroughly explored in previous
machine learning research. Our research has identified a set of constraints
for systems which are real-world learners and classifiers. These constraints
are inherent in the representational demands of natural and human goal-
directed concepts, the conditions under which classification is typically per­
formed, and the experience (i.e., training) available for learning. The design
of a successful system must satisfy the constraints of each.
1.3.1 Representation
A system must not make representational assumptions which are incon­
sistent with the concepts that it is intended to acquire. Natural and